Method for dynamically adapting user interfaces with changing user attributes

ABSTRACT

A method of displaying information of interest to a user as stored in an attribute database. The attribute database is a database of parent and linked child objects that has data about a user&#39;s characteristics and interests (content, commercial products, services, etc.) learned over time. A presentation database of parent and child objects is created from the objects in the attribute database using presentation templates to identify which of the attribute objects to use to create the presentation objects. The attribute child objects are sorted based on at least one trait in the attribute child objects and the presentation objects are linked together in the same order as the sorted attribute objects. The ordered presentation child objects are then linked to the corresponding presentation parent object and the selected presentation parent object and linked presentation child objects are displayed on a display.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 13/771,931, filed 20 Feb. 2013, which claims thebenefit of the filing date of U.S. provisional application No.61/668,054 filed 5 Jul. 2012 the contents of both are incorporatedherein by reference in their entirety.

FIELD OF THE INVENTION

The invention pertains to the field of information processing where datais collected or harvested from a variety of sources, including TV, web,movies, videos, mobile phones, and photos owned by the user, processedby the invention, and a new class of output is produced that addressesprivacy problems and provides new services to the user and the dataproviders.

BACKGROUND OF THE INVENTION

The convergence of TVs and boxes connected to the Internet has producedunexpected changes in consumer behavior. Many users consider Internetaccess to be a means to provide them with even greater choices forcontent—such as allowing them to switch to Over-The-Top (OTT) contentrather than watch traditional TV programming. Additionally, Internetaccess provides them with the ability to research, investigate, and findbargains for products and services that interest them. Users spend evenmore time “channel surfing”, jumping from content to content, or lookingfor inexpensive products or services matching their needs and budgets.By not “locking-in” to a specific channel or content stream for anextended time or hopping from one merchant to another, they are in factnot behaving the way service providers and advertisers expect themto—which minimizes the effectiveness of traditional advertisingcampaigns.

Advertisers want to maximize the use of the Internet portal available tothem. They push intrusive advertisements over the portal often based oncookies and information that they have collected from users activitieson their websites that are in effect “recommendations” for the user topursue an action. The user is forced to watch these advertisements ifthey want to see the desired content. These advertisements show up asbanners, pop-ups, or pointer position-based advertisements that aredifficult to bypass on the TV. These advertisements are in addition tothe campaign based advertisement the user is already forced to watch ifthey view the live TV content through the TV network portal.

Content providers use the same Internet portals to invade user privacyby recording user actions on their servers. They use this information tobombard the user with advertisements and explicit content or merchandise“recommendations” hoping for a sale. They also sell or share userinformation to varying degrees as well as their “recommendations” forcommercial benefit.

Although the recommendation provided by content providers may providesome benefit to the user, the recommendations are handicapped byfundamental limits on the “quality” of the data they use to provide therecommendations. The content providers use all the means currentlyavailable to them to record user actions. For business and legalreasons, this data is typically not shared with other content providersso there is no opportunity to compare and refine the accuracy of thedata based on a broader sampling. In order to increase accuracy, eachprovider “tweaks” or customizes their methods—which often runs upagainst the problem of invasion of privacy. There are many methods usedby content providers to determine “user preferences” from a distance.Regardless, the information deduced by a content provider at a distancewill generally be inferior when compared to the information that can becollected locally where the user operates; which is to say at the pointof origin of the user data.

Existing techniques for analyzing user preferences can be split into twobroad categories characterized by vendor or provider. The first categoryincludes vendors such as Amazon, eBay, and Netflix that offer commerceor content as a service or for sale. Such vendors develop and maintain“user profiles” based on: (1) what the users search for in merchandiseor content, (2) “visits” information such as reviews or productinformation, and (3) the activity recorded in their accounts. Usingthese profiles, vendors can make “suitable recommendations” in the hopeof increasing sales.

The second category includes vendors such as Microsoft, Google,Facebook, and Apple whose primary business is to dispense information orprovide services related to the information. Such vendors' offeringsinclude search engines, information databases, and applicationmarketplaces that users routinely access. These vendors desire todetermine and maintain track of user behavior so they can provide themost relevant and current information on a search query, fastest accessto the database, and the newest applications in their applicationmarketplace for sale.

All of these vendors employ techniques whose scope is limited to whatthe user does on or through their websites. They use sophisticatedalgorithms that rely on information such as, how a user initiates thesearch subject and the key words used, navigation over the resultsoffered, position of the mouse pointer, how much time the pointer spendson a particular position, the number of clicks on links formed within achain of user's (member's friend list) and so on.

User devices currently available in the market (e.g., Set-Top-Boxes(STBs), PCs, laptops, smart phones, tablets, and so-called “Smart TVs”),whether purchased through retail or supplied by service providers, aretypically configured by the manufacturer or retailer to go to themanufacturers or service provider's server on the Internet, the InternetProtocol (IP) based private network, or 4G—Long-Term Evolution (LTE)wireless network. When Internet is selected on a “Smart TV” purchasedfrom a retailer, for example, the User Interface (UI) displayed will bethe one supplied by the manufacturer. Similarly, when an IP STB providedby a TV service provider is turned on it will display the serviceprovider's UI. In both cases the UI is driven by a server owned byanother party whose interests might be different from that of the user.While the other party may be focused on driving the user to downloadmusic or movies from their “preferred” content supplier, that suppliermay not be the best fit to the user. This only offers limited choices toa user and does nothing to resolve the problem of user privacy or reducethe manual processing of information required to locate the bestalternatives available across the Internet that match the detailed andaccurate personal preferences of the user.

SUMMARY OF THE INVENTION

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

Described embodiments provide a method for displaying user-centric dataon a display. A presentation parent object is selected from apresentation object database having a plurality of presentation parentobjects therein. A presentation template corresponding to the selectedpresentation parent object is identified, the identified presentationtemplate having keywords therein and having one or more correspondinglower level presentation templates. An attribute object database issearched to identify an attribute parent object therein that matches oneor more of the keywords from the identified presentation template, eachattribute parent object linked to one or more attribute child objects.The identified one or more attribute child objects linked to theattribute parent object are searched to identify one or more of theattribute child objects that matches one or more keywords in the one ormore lower level presentation templates corresponding to the identifiedhigher level presentation template. Then the identified one or moreattribute child objects are sorted based on at least one traitassociated with each of the attribute child objects. One or morepresentation child objects are created from the one or more sortedattribute child objects. Next, the one or more created presentationchild objects are linked together in the same order as the one or moresorted attribute child objects. Then the one or more linked presentationchild objects are linked to the selected presentation parent object, andthe selected presentation parent object is displayed on the display.

BRIEF DESCRIPTION OF THE DRAWING

Other aspects, features, and advantages of described embodiments willbecome more fully apparent from the following detailed description, theappended claims, and the accompanying drawings in which like referencenumerals identify similar or identical elements.

FIG. 1 shows a block diagram of a system operating in accordance withexemplary embodiments of the invention;

FIG. 2 is a functional block diagram of the system of FIG. 1 accordingto at least one exemplary embodiment of the invention;

FIG. 3 is a simplified flowchart of a Attribute Repository managementprocess;

FIG. 4 is a simplified flowchart of a user actions and Search FlowDiagram process;

FIG. 5 is a simplified flowchart of a user initiated search process;

FIG. 6 is a simplified flowchart of an expansion of the search processof FIG. 4;

FIG. 7 is a view of an exemplary subcategory display;

FIG. 8 is a view of an exemplary level 1 and level 2 subcategorydisplay;

FIG. 9 is a simplified flowchart of a user interface actions;

FIG. 10 is a simplified flowchart illustrating use of Menu buttonoptions in the user interface;

FIG. 11 is a simplified flowchart of a Presentation Repository updateprocess;

FIG. 12 is a simplified flowchart of an process for creating a new user;

FIG. 13 is an image of an exemplary user interface overlaying a TVprogram image;

FIG. 14 is a block diagram of a mobile device for operating inaccordance with exemplary embodiments of the invention;

FIG. 15 is a functional block diagram of the system of FIG. 14 (mobiledevice) according to at least one exemplary embodiment of the invention;

FIG. 16 is a state flowchart of a device being acquired in amulti-device environment; and

FIG. 17 is a state flowchart of synchronizing a device in a multi-deviceenvironment.

DETAILED DESCRIPTION

Reference herein to “one embodiment” or “an embodiment” means that aparticular feature, structure, or characteristic described in connectionwith the embodiment can be included in at least one embodiment of theinvention. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment, nor are separate or alternative embodiments necessarilymutually exclusive of other embodiments. The same applies to the term“implementation”.

It should be understood that the steps of the exemplary methods setforth herein are not necessarily required to be performed in the orderdescribed, and the order of the steps of such methods should beunderstood to be merely exemplary. Likewise, additional steps might beincluded in such methods, and certain steps might be omitted orcombined, in methods consistent with various embodiments of the presentinvention.

Also for purposes of this description, the terms “couple”, “coupling”,“coupled”, “connect”, “connecting”, or “connected” refer to any mannerknown in the art or later developed in which energy is allowed to betransferred between two or more elements, and the interposition of oneor more additional elements is contemplated, although not required.Conversely, the terms “directly coupled”, “directly connected”, etc,,imply the absence of such additional elements. Signals and correspondingnodes or ports might be referred to by the same name and areinterchangeable for purposes here. The term “or” should be interpretedas inclusive unless stated otherwise.

Moreover, the terms “system,” “component,” “module,” “interface,”“model,” or the like are generally intended to refer to acomputer-related entity, either hardware, a combination of hardware andsoftware, software, or software in execution. For example, a componentmay be, but is not limited to being, a process running on a processor, aprocessor, an object, an executable, a thread of execution, a program,and/or a computer. By way of illustration, both an application runningon a controller and the controller can be a component. One or morecomponents may reside within a process and/or thread of execution and acomponent may be localized on one computer and/or distributed betweentwo or more computers.

Table 1 defines a list of acronyms employed throughout thisspecification as an aid to understanding the described embodiments:

TABLE I USB Universal Serial Bus STB Set-Top-Box OTT Over-The-Top IPInternet Protocol CPU Central Processing Unit IPTV Internet ProtocolTelevision 3G/4G Third/fourth generation mobile communicationstechnology DLNA Digital Living Network Alliance ATSC Advanced TelevisionSystem Committee DVB-XX Digital Video Broadcast, where XX = T, T2, . . .for Terrestrial Service SW Shortwave AR Attribute Repository uPnPUniversal Plug-and-Play COn Child Object, level n WiFi Wireless FidelityFIFO First-In, First-Out I/O Input/Output PC Personal Computer UI UserInterface USB Universal Serial Bus SEO Search Engine Optimization TCP/IPTransmission Control Protocol/Internet Protocol HD High Definition FMFrequency Modulation MW Medium Wave LW Long Wave PR PresentationRepository EPG Electronic Programming Guide URL Uniform Resource LocatorTCP/UDP Transmission Control Protocol/User Datagram Protocol

The described embodiments addresses several problems with priorapproaches to user searches of the Internet, including: (1) thedifficulty for users to obtain content, information, and deals thatmatch their interests, (2) the need for users to sift through largeamounts of mostly irrelevant search results provided by Internet searchcompanies, (3) preventing Internet service companies from invadinguser's privacy, (4) the limited ability of websites to provide userinterest relevant recommendations, and (5) the virtually non-existentability of a website to obtain recommendations from other websites auser visits. It is also desirable to curtail user annoyance anddisaffection caused by intrusive advertisements that are generallyirrelevant to the user.

The described embodiments focus on collecting the broadest and accurateinformation about the user and keeping it in a secure environment whereit can be used to develop the right processing framework for theservices that the invention provides. The point of origin of the userdata includes his/her home, places of travel, and workplace. Devicesoperating in this “user domain”, which includes set-top-boxes, PCs,laptops, smart phones, tablets, and Smart TVs, need to be smart andpowerful enough to process information and generate the desired outputwhile maintaining privacy and preventing external intrusions.

The invention might be embodied in software that can be implemented onany device or combination of devices operating in the user's domain thathas the ability to compute and store information. It addresses theproblems listed above and might provide the following benefits to theuser, content providers, and advertisers:

For a system or device to act as an agent for a user, it is desirablefor the system to anticipate and learn a user's needs and likes and,based on this, to identify the decision-supporting pieces of informationout of the huge amount of data an Internet search or web surfingprovides (site recommendations, numerous reviews/blogs, advertisementsetc.). The decision-supporting pieces of information relative to whatthe user is watching or investigating at that time are generallyrelevant and are presented to the user. This type of intelligentcapability reduces the time and effort for technical and non-technicalusers alike by using adaptive algorithms that continuously refine thereceived data.

Guided by this pertinent decision-support information, a user is muchmore likely to take action benefiting the content provider, a commercialestablishment or an advertiser by making a purchase, watching content,or “asking” to view an advertisement.

The software runs on a box inside the user's home and on a platform(s)in the user's possession, such as a smart phone, with the user havingfull control over its actions and contents thereby maintaining privacy.

The machine-learned information is compartmentalized by the inventionwithin the user's device in “buckets”, also referred to herein as anAttributes Repository, that support the device specific (TV, mobile)watching experience. The relevant data in these buckets is used topresent user interest centric information to be matched with product orservices at the appropriate time. The appropriate time is determined bywhat the user is watching or investigating or, in case ofadvertisements, suitably scheduled at times that is least intrusive tothe user and when the advertisement can be most effective. This reducesthe ability of service providers, content providers, and advertisers toforce irrelevant or badly-timed interruptions on the user. The user'soverall watching experience is improved while reducing the incentive forthe user to abandon a channel and its content.

Data in the buckets, suitably filtered, is sent to the serviceproviders, content providers, and advertisers to enable them to offersuitable merchandise, content, or advertisements to the user. Individualcustom Attribute Filters in the device, which can be configureddifferently for each service provider, content provider, or advertiser,are provided enabling the data sent to that destination to be suitablyfiltered preserving user privacy.

A suitable User Interface (UI) is provided that enables selection ofcategories, such as Broadcast TV, Movies or Shopping, and content suchas a specific TV network in the Broadcast TV category, using a simplepoint and click remote control or other input means. This approach tonavigation enables non-technical users to benefit from the advancedintelligence provided by a system incorporating the invention thatsupports the UI. The system learns from the content or by searchperformed by the user. Since the invention's processes are adaptive, thesystem will take a little more time to learn from a less sophisticateduser, who may input vague Internet search terms, when compared to themore sophisticated user, who will input more refined search terms.Nevertheless, given enough time, the device will gain enough knowledgeto converge and act as an agent for either type of user.

Although the disclosed embodiments are initially targeted at TVwatching, it can be deployed on any equipment that has the ability tocompute and store information and is used in other businesses. Thecategories in the invention can be extended to cover a broaderknowledgebase for the general consumer or decreased to support anarrower and deeper knowledgebase when deployed by, for example, abusiness. For business applications, the system learns within the scopeof the business categories and executes searches and display searchresults limited to these categories. The system would continuously learnby combining information from multiple websites and user inputs, therebycombining the search criteria and filter the results, specific to thebusiness, in a way unavailable with today's generalized search products.Additionally, Attribute Filters would return the relevant customermetrics based on actual user behavior for as business application

The disclosed embodiments support collaboration with cloud, mobile, orother devices when owned by the same user or where the user has accessthrough an account or other means. In this implementation, a lightweight“client” component of the Application and data objects are installed onthe mobile or other devices to provide all benefits of the invention tothe user when, for example, a mobile device is traveling. The clientversion of the data objects is tailored for the device usage. When thatdevice is in the vicinity of the device running the primary application(typically a STB box and also referred to herein as a hub device), itconnects with the hub device over a communication link or other usernetwork. As will be described in more detail below, the mobile devicethen synchronizes with the hub device and exchanges data to update theuser's interest information in the hub device, such as location-basedpreferences and services. After synchronization, the hub and the mobiledevices are up to date on user preferences and continue to learn withusage.

The disclosed embodiments might also allow the user to synchronize allcontent and subscriptions owned by the user across all devices owned orcontrolled by the user. Such collaboration eliminates the need to havecopies of contents on all devices saving storage space or obtainingduplicate subscriptions to watch content while traveling. Content on anydevice can be accessed by any other device and a subscription throughone device can be accessed by all other devices. Attribute Filters areused to set up a trusted link between two devices for content transfer.For example, if a user is in a foreign country, he or she can accessvideos, music, photos, and data files on his hub device or his home PCthat are not on the mobile device. Similarly, a local television channelor an OTT movie could be accessed by the mobile device through the hubdevice that has subscriptions to both.

In the embodiments described herein, user characteristics are createdand stored in the device based on his or her direct input and activitieswith the preferences based on actions the user has taken or is taking onthe responses. This might produce more accurate results than existingtechniques that determine the characteristics of the user from adistance.

The prior techniques determine the characteristics of the user based onuser access of one vendor's website making all results andrecommendations skewed towards this particular vendor. The embodimentsherein might access data from multiple websites and filters searchresults to create a profile that best serves the user.

In order to increase advertisement “accuracy”, vendors are continuouslytrying to access more information about the user. This has resulted invendors encroaching on user privacy, sometimes disregarding user privacysettings. Another disturbing trend is vendors selling user's secureinformation to other vendors. The embodiments run as a customapplication (separate from the standard browsers and operating systemsthat can be “hacked”) on the device with the user controlling theprivacy and security. Search requests are transmitted to multiplewebsites and the results extracted. Information verified by the user andenabled through Attribute Filters is transmitted to vendors and serviceproviders. The user receives the information without compromising theirprivacy.

Currently, content and merchandise providers have the upper hand inbusiness negotiations since they collect user information and caninfluence the merchandise and the contents “recommended” to the user.Infrastructure entities such as telephone and cable service providersare relegated to operate as “dumb pipes”, i.e., simply acting asconduits for content and advertisements. By providing control of whatusers watch and buy, the embodiments herein might enable them to seekand obtain content or merchandise from any provider or a merchant. Thisreduces the control that a particular vendor currently enjoys over theuser and the service providers.

Generally, service providers negotiate carriage rights for content withthe studios, content creators, and distributors. The price serviceproviders pay provides them with the leverage that they use to price thesubscriptions that users pay as well as the advertising rates that theadvertisers pay to broadcast advertisements on the service provider'snetwork. This creates an environment in which the service provider setsthe price of the subscriptions, bombards the user with advertisementsbased on what the advertisers are willing to pay (hereafter called “adcampaign”) and, on a very limited scale, promote the merchant with whomthey have business deals. Instead, a framework is created where usersare favored for the best deals in content and merchandise and hence areless sensitive to the subscription price a vendor can set for content orto a merchant promoted by the service provider. By employing the storeduser characteristics, the annoyance of ad bombardment “pushed” by the adproviders onto the user is reduced and the user can “pull”advertisements from ad providers based on the stored characteristics.

The techniques and system described herein might move influence towardthe broadband provider and user to provide “negotiation power” toservice providers as they supply broadband access to homes and users.Should the service provider choose to provide a device incorporating thetechniques described herein, they might be able to obtain filteredinformation about content and merchandise with the user's permission.Armed with a large subscription base and the information, serviceproviders could negotiate favorable carriage rights deals with manycontent and merchandise providers benefitting significantly from theupside.

Referring to FIG. 1, an exemplary block diagram of a physical system 100similar to a Set-Top-Box (STB). The system might be a stand-alonesystem; devices embedded in a TV, Home or Residential Gateway, or existin some other form. In one embodiment, this box contains: (1)Input/Output port(s) 102 to send or receive video data and otherinformation through private or public network, (2) a computationplatform or CPU 104 to run the processes, (3) a storage device 106 tohold user and computation data, and (4) an Output(s) 108 for display orto send information to a TV monitor. Additional hardware, such as systemmemory 110, an optional off-the-air or cable tuner 112, USB or otherports 114 for interfacing various devices, such as flash memories, andother input/output ports 116 for interfacing with a keyboard, mouse,etc. All of these are interconnected by structure 120, such as a bus.

In a first implementation, the source of videos and information isBroadcast TV (TV channels received by means of a tuner 112), InternetTV, video websites accessed through a broadband Internet connection toInput/Output port 102, and networked personal devices (media servers,phones, cameras, etc.) coupled to the system 100 by, for example,through the networked Input/Output port 102 or the USB ports 114. Othersources of video and information delivery, such as IPTV or mobile 3G/4Gaccess, can be provided and do not change the inner workings of thesystem 100.

In one embodiment, the system 00 operates on a “per-user” basis. A usercan be a group, such as a family, or a single person, such as a fatheror a mother. As used herein, the components that hold (or store) keyinformation that drives the processes are discussed as one per user.FIG. 12 later describes the creation of a new user in conjunction withthe user interface (UI).

If the system 100 is implemented in a gateway or other physical systemthat has sufficient computing power and storage and is thus able tosupport multiple users with TVs, computers and other display devices,then the physical system can support multiple virtual single usersystems enabling multiple users to be supported at the same time.

FIG. 2 is a functional view of the system 100 of FIG. 1. The functionalsystem 100 shows the function of the various components of FIG. 1,described below in the various embodiments, connected via numbered flowlinks.

In FIG. 2, a User Input 202 flows through flow link 204 connected to theUser Interface (UI) 206 through which commands are sent to the system100 using devices coupled to Input/Output (I/O) ports 102, 114, or 116in FIG. 1. These devices may include remote control, keyboard, mouse, asmart phone or a tablet capable of issuing commands, a voice recognitionmodule for voice based commands or a gesture recognition module forgestures based commands. Devices, such as smart phone or a tablet, havescreens that can display commands and status in addition to issuingcommands through touch screen or a virtual keyboard. In FIG. 1, thesedevices can physically interface with the system 100 through anyinput/output port.

FIG. 2 also shows content, such as video or Internet data, passingthrough flow link 208 connected to the system 100. The system receivescontent and sends requests and other communications using this link 208.The content can be any type of data including audio, video, streaming,text, or RF data that complies with the applicable interface standards.The standards include, but are not limited to, ATSC broadcast TV, DVB-XXTV, radio (MW, LW, FM, SW, HD, and Satellite), Internet protocol (IP)based TV, Cable TV, Internet Browser's supported protocols, and manymore. In FIG. 1, all such content can come through any of the ports 102,114, or 116.

The system 100 has an output port 108 through which content or commandsare displayed to the user on a display 122. FIG. 2 shows the outputpassing through flow link 210 connected to a TV, such as monitor 122.The display can also be a screen if the system as defined here isimplemented on the hardware of the display device such as a Smart TV ortablet. The system output through flow link 208 can also be to a networkthrough any of the ports 102, 114, or 116 in FIG. 1.

The Application 212 in FIG. 2 is the process that sends and receivescommunication to the user through User Interface 206, receives contentand sends request and network requested information through link 208,and displays output on the screen through link 210. The process usesAttribute Repository Structures 220 and Presentation RepositoryStructures 250 stored within the system 100 to perform computations andmanage all communications.

The system has two primary modes of operation. They are: (a) Learn mode,and (b) Convey mode. Both of these modes use structures labeled as UserInterface, Application with processes, Attribute Repository Structuresand Presentation Repository Structures shown in FIG. 2.

Learn Mode

In Learn mode, the system 100 uses structures 220 and processes to learnas much as it can about the user. In Convey mode, described later, thesystem uses all the information it has learned and stored up to thatpoint in 220, matches it to the user, and using information previouslystored in 220 obtains new information, best deals, and fetches or pullsin advertisements. The results are then displayed to the user asrelevant content, results, and deals, and are more relevantrecommendations that are not confined due to being obtained from asingle website or a provider.

Consider a scenario in which a user is watching TV. The user uses aremote control to switch to another source of content, e.g., switchingfrom an on-the-air TV broadcast to a streaming video from a websitethrough the Internet. For this discussion, assume the new content is amovie with a well-known artist, such as “Famous Artist 1” as a star.

Referring to FIG. 2, the remote control input is received by the UserInterface (UI) block 206 over flow link 204. The UI block 206 causes theApplication block 212 to start through flow link 214.

The Application 212 determines what needs to be displayed to provide theuser the desired programming. This is described under Convey Mode laterin this document.

In the Learn Mode, the Application 212 uses blocks in box 220 labeled“Attribute Repository Structures”. The first block is AttributeRepository (AR) 222 that contains Parent objects labeled AR-1, AR-2,etc. Linked to a Parent object AR-2 through link 225 is the Child objectCO1-1. Linked to the Child object CO1-1 is another Child object CO2-1linked through link 230. The links 228 and 230 are forward and backwardlinks. Each AR object could have multiple links. Multiple links meansthat the AR objects points to more than one child object list. TheAttribute Repository Structures 220 might be implemented as a databasewith hierarchical objects residing in the system or in another location.It might also be implemented as a collection of files and directories.Other structures within 220, namely Seed 224 and Attribute Filter (AF)234 might be implemented as database or a collection of files anddirectories.

A Parent AR object, e.g., AR-1, AR-2, etc., in AR 222 describes acharacteristic or interest of the user that matches with how the productwill be used. In one example, the system 100 is used as a television.The things a television is most suitable for at home are called its“usage classifications”. The initial focus for TV usage classificationis entertainment, commerce, social media, and advertisements. The Childobject CO1-1 describes a property of one of the usage classificationsthat is linked to a Parent object AR-2 in 222. As an example, if Parentobject AR-2 represents a user characteristic of movies owned or liked,then the Child Objects CO1 might represent a list of the movies the userowns or had watched with CO1-1 Child Object describing the first movie.Child Object CO1-2 describes the second movie, and so on. Child objectCO2-1 represents a second level characteristic of Child object CO1-1,linked through link 230 that is one of the “items” about the movieCO1-1, the user owns or likes. The Child Objects are created for itemsuser has shown interest in through explicit searches, clicking onrelated items, and so on. Child Object CO2-1 may represent an artist inthe movie the user likes, CO2-2 represents at location where the moviewas filmed and the user has shown interest in places, CO2-3 mayrepresent a product that was shown in the movie and the usercharacteristics show inclination towards such products. AR Structures220 does not show it, but it is easy to see that another level of childobjects linked to any one of the second level child object (AR Structure220 shows CO2-1, . . . ) would further expand on the items reflected inthe second level child object. This can go on until an explicit product,service, or an advertisement has been identified. The linked AR objects,beginning with a Parent object AR-2 and terminating at a last level ofchild object (FIG. 2 shows CO2-1), forms a chain that matches usercharacteristics with the lowest level of detail the system 100 has forthe usage classification. The Child objects such as CO1-1, CO2-1 arelinks in the chain and as stated earlier, have levels. Level is thedegree of how far removed an AR object is from the user characteristicsrepresented by Parent AR object AR-2, and defines the “progress” thesystem has made in matching the user characteristics to the usageclassification. A list of words and phrases that facilitates matchingthe characteristics and properties of the user and products, services,social media and the advertisements at each level is provided in theSeed 224. The Seed 224 as seen in FIG. 2 is divided into compartments.Each compartment corresponds to a level in the Attribute Repository AR222 consisting of Parent and linked Child objects. The first column 238in the Seed 224 contains a set of words that defines usercharacteristics suitable for creating a Parent AR Object in thedatabase. Other columns in the Seed 224 contain words that are suitableto create next level down (e.g. moving left to right) Child AR Objects.The algorithm in FIG. 3 identify “pertinent to content” and “close touser interest” words or phrases when it extracts information fromaccessed web pages and generate objects that are then stored in the ARStructures 220 database. The keyword that describes the characteristicsis stored in the newly created AR object as well for match whenalgorithm in FIG. 3 and FIG. 4 are executed again.

The usage classification of a product based on system 100 also dictatesthe user interface (buttons, choices menus, etc.) options presented tothe user. During the learning process, each Attribute Repository objectis created or updated with words or phrases defining characteristics,links to other AR objects, level in the chain, time when created, visitcount and other such parameters. One such parameter is called the UIcategory tag. This tag is a constant and indicates where in the UIlayout, consisting of categories with their own defined levels, would aparticular AR object reside, should it be part of the UI. The UIcategories are shown in FIG. 7 and FIG. 8. The UI category tag isprovided through the Seed 224. Each word or phrase in each compartmentin the Seed that matches the UI category is provided with the UIcategory tag. UI category tag in the AR objects improves the efficiencyof the algorithm that correlates the AR objects with the PresentationRepository (PR) objects shown in the Presentation Repository (PR)Structures 250 in FIG. 2. The algorithm, shown in FIG. 11 and describedin more detail below, updates the PR objects in PR Structures 250 basedon the attributes in the AR Structures 220 in FIG. 2 so that the UIdisplay better reflects the user's interests and preferences. In theLearn mode, the Application 212 looks at what was selected by the userthru the UI, for example “Movies”, which as described later is a UImain-category item. Based on this, the application looks intoinformation such as meta-data typically provided by the streaming serveror movie database, or, if the content is streamed, other information andlinks on the page sent to the system 100 by the server. Extraneousinformation that is likely to be of questionable relevance or unrelatedto user activity such as advertisements, promotions, and other suchitems are not used in this analysis. The key words and phrases areextracted and then analyzed by the Application 212 to determine itsassociation with the user's interests.

To demonstrate association with multiple user's interests, Application212 uses flow link 226 to find an Attribute Repository objects in theAttribute Repository 222 that match, or are semantically similar innature to, the keywords or phrases the Application 212 has extractedfrom the content on the screen. When an AR object is created, thekeywords responsible for the creation (user characteristics, product,advertisements, or service properties, etc.) are stored within. After amatch is found between the keywords, the application uses forward andbackward links, such as 228 and 230, to locate other linked Parent orChild AR Object(s). The linked objects may be part of a list, such asCO1-x and CO2-y shown in FIG. 2 where “x and y” are the position in theChild Object CO1 and CO2 list respectively, or a different Child Objectlist, such as Child Object CO25 (not shown), that through links, leadsto a different parent object AR35 (not shown) in AR 222. As time passes,there could be more AR objects linked to each other singularly, meaningthrough a single forward or backward link, or to multiple objectsthrough multiple links. When an AR object is linked to multiple childobject lists, it means that the information described by this AR objectis “related” to other pieces of information described by the linked ARobjects. As an example, consider the parent AR object AR-2 in 222 thatrepresents “movies”. The linked Child Objects (CO1-1, CO2-2) leads toanother Child Object CO25-5 (not shown in FIG. 2) that identifies alocation where the movie was filmed, e.g. United Kingdom. This set oflinked AR objects, from AR-2 to CO25-5, represents one chain. A secondset of forward and backward links in CO25-5 may point to a Child ObjectCO12-3 (not shown) which represents royalty. A different backward linkfrom Child Object CO12-3 points to a parent AR object AR35 (not shown)in 222, which represent vacations. This set of linked AR objects, fromAR35 to CO25-5, represents a second chain. Both these chains have oneshared Child AR Object, CO25-5 that represents a location (UnitedKingdom). The chains enable the system 100 to associate usercharacteristics of movie with vacation through a common location, UnitedKingdom. Thus, by following the AR objects through links and formingchains, the system can form associations with all related usercharacteristics, and content, products, services, advertisements, etc.

In the example of movies with Famous Artist 1 as a star, the simplestway to describe creation of the AR structures in 220 and how they relateto the PR structures in 250 is if the AR parent object, e.g., AR-2 isMovies which also is as UI main-category item. The Child Objects CO1-1would include the first movie the user owns or has watched that has thehighest visit count. The child object list CO2 would contain informationabout the movie that the system learned and correlates to interests ofthe user. Such information may include: Famous Artist 1 name in ChildObject CO2-1, director name in CO2-2, hook in CO2-3, location in CO2-4,products endorsed in CO2-4, and so on. A child object list CO3 (notshown), that is linked to the Famous Artist 1 Child Object CO2-1, maycontains additional information about the artist that interests theuser. Such information would include personal information in ChildObject CO3-1, such as age, height, weight, and other personal metrics.If the user is interested in family, a new Child Object CO3-6 (notshown) would be created and linked into child object CO3 list. A childobject list CO4, linked to Child Object CO3-6 (not shown and reflectingfamily) would then contain the family information. Child object CO1-2 isan object that is linked to CO1-1 and points to the second highestvisited movie. For each CO1 entry, the corresponding CO2 entries containlower level objects that points to additional information. The system100 creates child objects linked to each other within the same level(CO1-1, CO1-2 . . . ) and at different levels (such as, CO2-1) as itlearns more about the user and the usage classification. A farthestlevel child object entry is created that contains an Internet link thattypically completes the chain that links user characteristics with aparticular product, service, or advertisement. The system 100 does notstore any detailed information internally as the information can becomestale. Generally, the newest and the most up-to-date information areprocured using the Internet links.

To illustrate a different way of creating and linking the structures,the Parent object AR2 could represent “Favorite Artists”. In such case,the entry CO1-1 would contain the name of Famous Artist 1 as the mostvisited artist. The entry CO1-2 would be the next most visited artist,such as Famous Artist 2, and so on. Child object CO2-1 for Famous Artist1 would contain other items that are specific to the artist and werefound to match the user's interests for which another AR object exists.Example of one such interest would be fashions. Others include: Movies,Family, Charities, etc. represented by Child Objects CO2-2, CO2-3, andso on. Each one of these items would have a link that may point tothird-level Child Objects CO3. As an example, the third level ChildObject for movies would have CO3-1 object reflecting the most visitedmovie starring Famous Artist 1, the Child Object CO3-2 would have thesecond most visited movie starring Famous Artist 1, and so on. Eachmovie Child Object would have links that would be pointing to the fourthlevel Child Object CO4 chain representing information about the moviethat the system learned and which correlates to the interests of theuser. With such objects and links, the system essentially have developedmeans to link interests of the user with content, products, services andadvertisements that the user may be interested in.

The size of storage 106 in FIG, 1 available in system 100 and how muchof this storage is required to run the functionality of a STB or agateway determines how much storage space can be allocated to structures220 and 250 in FIG. 2. The limit on the size of structures 220 and 250is preset by the vendor who makes the product, such as a STB, that usessystem 100. When the system 100 reaches the limit of the storageallocated for structures 220 and 250, it can, (1) “age out” the oldestentries in structures 220 and 250 and use that space to create new ones,(2) it can archive the older entries to cloud storage or other networkedstorage and retrieve them when needed, or (3) it can request user tomanually increase the space or discard some entries. Other methods canbe used. In one embodiment, the Application 212 uses method (1) where itautomatically removes the older entries to create new ones.

As described above, the Application 212 begins the process over flowlink 226 by matching keywords it has stored in all the AR objects in theAR Structures 220 with words in the meta-data associated with content orthe web page on the screen. This matching enables the Application 212 toidentify user's characteristics that is pertinent to content or the webpage on the screen and to identify other information that can be smartlypresented to the user. If a match was found, the Application 212 hasfound something about what the user is currently seeing on the screenand associates that with information it has already learned. If amatching AR object was not found, the Application 212 determines “closeto user's interest” information from what is being displayed on thescreen which will then cause it to create a new AR object in ARStructures 220.

Not finding a matching AR object in AR Structures 220 means that thesystem 100 does not have knowledge or information about what is beingdisplayed on the screen. The Application 212 uses flow link 226 and 232to use the Seed 224 to match any of the words in the Seed with words inthe meta-data for the content or the web page on the screen. If any ofthe words in the Seed 224, in any of the compartments or AR child objectlevels, the first of which is labeled 238, matches the words in themeta-data or the web page on the screen, then the meta-data or the webpage has information that matches the usage classification provided forthe system 100. In the example, the usage classification is a TV. Amatch with the Seed indicates that an AR object can be created in the ARStructures 220. If there was no match found between any of the Seedwords and the meta-data associated with content or the web page on thescreen, then the Application 212 uses natural language tools availableover the Internet or as separate application to determine keywords inthe meta-data or the web page on the screen and then compare thesekeywords with words in the Seed 224 that are semantically equivalent todetermine if the new keyword would create a parent AR object or as childAR object. Once the corresponding AR object is created, subsequentsearches would create its relationship (such as links) with other ARobjects as the system continues its learning process. After creating theAR object, the new keyword is added to the Seed in the compartment thatis semantically equivalent to the new keyword. Using the Seed in thisway might decrease the time required to determine a new characteristicthat has not yet been entered in the AR as an object. Natural languageand semantic determination takes longer and is the way the system willdetermine a “new characteristic” of the user or learn about a product ora service if one was not found in the Seed.

Attribute Fitter

The Attribute Repository Structures in box 220 also show an AttributeFilter (AF) block 234 with objects labeled as AF-1, AF-2, etc. Overtime, the Attribute Structures 220 would contain considerableinformation about the user and about products, services, advertisements,etc. Some of the user information system 100 has learned will includepersonal information such as accounts, personal preferences, financialinquiries, behaviors, religious or sexual bias, favorites, etc. Theinformation can be displayed to the user to show what the system haslearned. Limited information, such as account sign-in, products orservices searched, is conveyed to the outside world to request content,more information, or to provide analytics data to vendors and serviceproviders as part of additional service or as some sort of a businessarrangement such as a subscription or discount. The amount ofinformation sent for display or to external vendors is controlled by theAttribute Filter 234.

The Attribute Filter 234 is provided with key objects by design based onthe product usage and the business arrangements. The businessarrangements define the communication objects developed based on thebusinesses or service provider's needs and the services they offer. Auser might be able to modify permissions saved in the AR objects in 222as one of the parameters, add, or delete communication objects (i.e.protocol packets parameters) in the AF 234. Display and certain minimumpersonal information objects, in one embodiment, cannot be deleted.Control of object properties is provided for those objects that can bedisplayed or used for external communication or both. The Application212 communicates with the Attribute Filter over flow link 236 in FIG. 2.

Examples of AF objects include:

(1) Attribute Repository pass-thru for display (AF-1). Generally, thisobject is not to be communicated externally. It provides parameters thatpermit display for all Attribute Repository objects in AR 222 and theChild Objects (CO1-1 . . . ).

(2) Search Result pass-thru for display. Generally, this object is notto be communicated externally. It provides parameters that enablelisting of the search results by order as determined by the AttributeRepository.

(3) Seed pass-thru for display. Generally, this object is not to becommunicated externally. It enables display of Seeds 224 so they can bemodified, added or deleted.

(4) Personal Information External. This AF object lists personalinformation parameters that can be retrieved from the AttributeRepository and communicated to the outside world. This AF object'sparameters can be added or deleted.

(5) Sign-in External. This AF object provides parameters that can beused to identify personal information in the AR personal informationobject that can be used to sign-in on the web, a cloud or a network.

(6) Business Object X.y. This AF object provides an internal ID “X”,such as a reference to an AR object that contains the name of a businessor a service provider, for validating a business name whose parameterscan be communicated externally. The “y” defines the type of parameterthat can be communicated. Examples of types of parameters include:number of times content browsed, number of times advertisement browsed,products purchased from business, etc. Other parameters may request someuser information. That information will be qualified with the PersonalInformation External AF described above. If the user has not givenpermission, the information will not be communicated externally.

(7) Ad Object X.y. This AF object provides an internal ID “X”, such as areference to an AR object that contains the advertisement info, forvalidating the advertisement whose parameters can be communicatedexternally. The “y” defines the type of parameter that can becommunicated. Examples of types include: number of times advertisementpresented, number of times advertisement clicked, vendor site visited,vendor agent visited, product or service purchased, etc.

An example of an Attribute Filter 234 usage is information needed tosign in to a particular website. Such information is sent over theInternet. An Attribute Filter will limit the information sent out to theminimum needed to sign into the account on the website. Examples of suchinformation are login id/email, password, gender, age/date of birth andlocation.

Other Attribute Filters are provided to pull in additional content fromexternal websites. One of these filters will cause the request for morecontent to have limited information made available to a provider (e.g.,a service provider, a content provider, or an ad provider) and specificto information being sought by the provider based on data in the parentor the child repository objects. An example of such a filter is arequest to an OTT movie provider like Netflix or Hulu, where the userhas an account, for the user's log-in information. Another example ofsuch a filter is a request for advertisements where the system pulls-inadvertisements from advertisement providers, and displays them to theuser at the user's “convenient” time. A separate process, shown in FIG.4, determines the convenient time to display the ad and the form the adis displayed in (banner, streaming or pop-up) based on its criteria. Anexample of a convenient time to play an ad is when the user has hit thepause button during a non-live video playback (at which time the user isnot focused on the content and would be more inclined to pay attentionto the ad and “click” on it).

Similarly, during live video playback, a system 100 process shown inFIG. 4 can override the inline advertisements and play theadvertisements based on the user's interests saved in the AttributeRepository as links. The advertisements to be played are accessed usingAttribute Filters 234. Such advertisements are much more likely to beeffective and generate meaningful “clicks” by the user, therebyproviding a higher return on investment to the ad provider. This mightbe done where detection of live and in-line advertisement stream “startand stop” is permitted.

Convey Mode

Now consider an example in which a user is watching TV and uses theremote control to switch to another source of content. The new sourcemight be another TV channel or a website.

In one embodiment, the system 100 presents a user interface navigationscreen on display 122 so that the user can point to and click on acommand button that offers the choices available to the user.

The system “creates” choices for the user by first learning about theuser's characteristics (Learn Mode) and then providing content,additional information, deals and recommendations based on what it haslearned. This results in a UI display in which the items such as thecontent of the buttons, listings, and results might be continuouslychanging.

The UI and its dynamic nature is managed and conveyed to the displaybased on the Presentation Repository Structures in box 250 shown in FIG.2.

Similar to the Learn Mode, the remote control input is received by theUser Interface block 206 over flow link 204. The UI block causes theApplication to start through flow link 214.

In Convey Mode, the Application 212 uses the Presentation RepositoryStructures block 250. The PR 252 is a database with Parent objects PR-1,PR-2, etc., and Child Objects PCO1-1, PCO2-1 . . . , etc. residing inthe system. It might also be implemented as a collection of files anddirectories.

The PR objects are related to the UI and describe a “display or ascreen” as it pertains to the usage of the end product, which in thisexample is television. Each UI screen, an example of which is shown inFIG. 13, might have multiple buttons, each represented by an object inthe Presentation Repository. For example, the display consists of itemssuitable for a defined usage, such as watching a TV show or a movie, andis facilitated by the PR objects.

The PR objects change based on the changes the Application 212 makes inthe Attribute Repository Structures 220 that reflect the user'scharacteristics and the interest the user has in content, products, andservices. A change in user interests in turn might cause new content tobe reflected in the buttons, the order in which they are displayed onthe screen, or the commands and navigation presented to the user whenbuttons are pressed on the remote control.

The Application 212 looks at the UI selection and, using flow link 254,accesses the Presentation Repository Structure 250. A PresentationRepository object exists in the PR as a Parent Object 252 that reflectsone of the main categories defined in the UI usage model. The ParentObjects are labeled PR-1, PR-2, PR-3, etc. As an example, forentertainment the parent objects might consist of TV, InternetTV,Movies, Music, Games, etc. For commerce, the parent objects mightconsist of shopping stores, auction houses, discounters, merchandiseprocessors, etc.

The Presentation Repository Structures 250 in FIG. 2 shows Child Objectsfor each parent object. Child objects are accessed using flow links 258and 260. After selecting the parent object, the application accesses theChild Objects using these links. These Child Objects lists are labeledas PCO1, PCO2, etc. There may be more (or less) Child Objects list perparent object.

Child objects contain the “next level down” information that is linkedto the parent object. In FIG. 2 PCO1-x is the first level Child Objectsand PCO2-y is the second level Child Objects where x and y is an integergreater than zero, e.g., 1, 2, etc.

In the movie example, the Child Objects can be used to reflect multipleways movie choices can be displayed on the screen. For example, thefirst level Child Objects PCO1-x might represent genre and the secondlevel Child Objects PCO2-y might reflect the movies available in each ofthese genres. In an alternative embodiment, the movies are organized bylocation where movies owned by the user are presented first (hard disk,DVD, Media server, etc.), followed by Streamed Free (user has accountsat places like Netflix, Hulu and so on), followed by Available for Sale(priced websites, DVD by price, etc.). In this case, the Child ObjectPCO1-x would show the listings criteria just described. Child ObjectsPCO2-y would then show the movies available under each of the listedcriterion.

The flexibility in the parent object and the corresponding Child Objectsand how they are structured provides multiple ways a display can be madeof UI choices without re-engineering the entire User Interface. If thereis only one Child Object list, then the display would be flat, i.e., thedisplay will consist of just one list. If there are multiple ChildObjects that are linked, then the display will reflect the depth innavigation, i.e., each Child Object corresponds to a navigation click onthe remote control. (For a television application, it might be desirablethat most user navigations conclude in less than four clicks.)

Similar to the Attribute Repository Structure 220 showing the Child ARobjects CO1-1, CO2-1 . . . , the number of Child Objects PCO and thedepth (how many objects in each child) might be controlled by a limit onsystem 100 the vendor might have set on the Presentation RepositoryStructures 250. The limit might be set by design considering the usagemodel, business needs, and the size of the storage provided in thesystem 100.

When new PR objects are created in the Presentation RepositoryStructures 250, as previously discussed, the new objects might requirenew items to be added to a UI navigation page or screen and suitablenavigation links defined in order for the newly created items to becomepart of the remote control operations. The Application creates the newentry using Template 256 and flow link 262. The Template 256 is apre-defined shell for the UI layout that is provided based on the usageclassification for the system 100. In the example, the usageclassification is a TV. The TV usage and the features it provides, e.g.watch broadcast or IP TV channels, InternetTV channels, Internet basedstreaming services, shopping portals, Internet searches, etc., defineswhat the UI would contain as buttons and the navigation. A possiblelayout of the UI and how the different buttons are placed in categoriesis described later with FIG. 7 and FIG. 8. The primary selection in theUI is provided in the main-category and corresponds to first compartmentor level 270 in the Template 256. Application 212 uses data in the firstcompartment 270 in the Template 256 to create Parent PR objects in PR252. The PR objects in 252 then show up as “main-category” buttons andnavigation in the UI. Further selection buttons on the UI within eachmain-category, called is sub-category, is provided through the ChildObjects PCO1-1, PCO1-2, PCO1-3, etc. that are created using lower leveltemplates shown to the right of parent PR object template 270 in theTemplate box 256. If the UI specifications require further selectionwithin the selection buttons represented by Child Object PCO1 list, thenthat selection, referred to as level-2, is provided through Child ObjectPCO2 list. Navigation links for parent and Child Objects are created bylinking the new parent object item in the Presentation Repository 252with the child items using flow links 258 and 260. If the PR parentobject already exists in PR 252, the child items are added by linking tothe new item in the Child Object list PCO1 and, if applicable, PCO2 asshown in FIG. 2.

Presentation Filter

The Presentation Repository Structures in box 250 also show aPresentation Filter (PF) block 264 with objects labeled PF-1, PF-2, etc.The Presentation Filter is provided to enable the application to make PRobjects in PR Structures 250, representing the UI items, that conform tothe screen of the attached display connected through flow link 210 (tothe monitor 122 in FIG. 1). This is accomplished through flow links 266and 268. For example, UI items displayed on a High Definition TV with16:9 aspect ratio would have a different layout compared to a StandardDefinition TV with a 4:3 aspect ratio. If the user interface is alsosent over a network as a separate data stream to be overlaid at a remoteterminal, such as when the system is operating as a gateway and a remoteterminal attached to a TV is performing the display, then the filterobject would contain properties of the interface of the attachedterminal. These properties would be used by the Application to send the“data packets” over the network to the remote terminal using flow link208.

The Presentation Filter 264 is provided with objects based on the typeof displays it is directly attached to, or the remote devices it cansend or receive data to render or the UI interface to overlay on thedisplay. The type of display can be determined by universalplug-and-play (uPnP) techniques or explicitly entered by the user if thedisplay is an analog TV set. In the event the system discovers a newdevice, the Application 212 uses a PF Template and fills it with newdevice particulars. It then stores the Presentation Filter object in thePF Block through flow links 266 and 268.

For a TV, the PF filter object could include properties such asresolution, aspect ratio, 3D capability, refresh rate, etc. For anetwork, the objects could include remote device identification, type ofinterface (TCP-IP, USB, DLNA, etc.), packet parameters, and so on.

Database

To implement the data structures described above and manage therelationships between them, a proprietary or a commercially licensableDatabase Management Systems (DBMS) such as Microsoft Access, SQL, or asimilar product, is used.

In FIG. 2, the AR and PR Repository objects consists of many Tables thatare linked together to form database objects and to form therelationships. The tables, their relationships, and the data (orrecords) are created and managed by the Application 212.

Although it could be implemented in a number of ways, in one embodimentthe description uses one database per user for simplicity and to manageeach user if the system has multiple users. In an alternate embodiment,one database is set up for the entire system and to provide access toeach user through individual user identifiers or User_IDs. The structureof the AR and PR would then have tables to access the right objectsusing these User_IDs.

Attribute Repository Structures Algorithms

The Attribute Repository Structures 220 is used to hold information onthe user's interests and on content, products, services, etc. that bestsuits those interests. The system 100 learns more about the user overtime as it tracks the user as he or she watches content or searches theInternet for information. If there is little or very limited user inputduring searches, and the display is primarily TV, most of theinformation contained in the repository will be entertainment objectsand biased towards TV. If the user has been looking for information inmany places, the information saved will be equally diverse.

In addition to what is described above. FIG. 2 also shows an exemplaryfunctional system diagram of the overall system and its operation withrespect to learning and display. The processes in the Application arenow discussed.

FIG. 3 is an exemplary flowchart 300 showing the various Applicationprocesses that manages the Attribute Repository Structures 250 andenables the system 100 to learn from the content.

Content received through one of the input ports in FIG. 1 is received bythe Application through flow-link 208 as shown in FIG. 2.

In FIG. 3, this content comes into the system at step 302, passesthrough Presentation Flow Process 304, and is displayed on the screen instep 306. The path shown is for flow links 208, 268 and 210 in FIG. 2.The Presentation Flow Process 304 consists of receiving the content,decoding it using a suitable decoder and then rendering it on thescreen.

Within the Application and for the Attribute Repository, contentidentification such as the URL, meta-data, and additional information iscached every time the content source changes. The content source couldchange, for example, if the user is flicking through TV channels,video-on-demand channels, or web pages, pauses for a few seconds toglance the content, and then continues to new content.

The process AR1 depicts caching to prevent incomplete updates orinstability if the previous update of the Attribute Repository is stillin progress.

In order to provide efficiency and identify the content that changed andgenerated user interest, each “changed content” is tagged with aclick-on and click-off time. The content with the longest watch-time(click-off minus click-on) generated user interest that would be used to“learn” about the user and the content. Content with very long playtime(e.g. movie) will be updated once after the previous AR update completessince the system did not register a content “change”.

If more computing power and larger storage is available, the system cancache more content with ever-shorter watch times until it reaches aScreen-Scan time. Screen-Scan time is defined as a time when a humangets turned off by just glancing at the content and clicks to anotherchannel. This time is learned by the system. Content in the cache withWatch-time less than the Screen-Scan time is not used for learning.

Process AR2 in further analyzes to determine if the content passed fromprocess AR1 has more focused user interest by determining, if the pageis being scrolled, where the pauses are to determine user's interest andthe position of the pointer which may indicate which content made theuser curious. In the future, user facial and gesture recognition, eyetracking, etc. can be used to determine likes and dislikes in a similarfashion. The process “narrows” the target content that will be used todetermine the user's interests.

Process AR3 takes the content that could potentially be interesting tothe user and extracts more details from it. If the content is live suchas TV or IPTV, additional information is obtained from the meta-data orextracted from the Electronic Programming Guide (EPG). If the content isfrom a web page, then a succession of techniques is used to quicklydetermine the words and phrases that would match user interests. First,screen-scraping technology is used to determine the main content such asvideo, text, pictures and games, etc. Other items on the screen, such asbanner advertisements, promotions and pop-ups are eliminated from theprocessing since they are not the main subjects to determine user'sinterests. Secondly, the words in the Seed 224 are compared againstwords in the main content to locate the matching to user interests'words. In step 320 if there is a match, the system learns that the maincontent is useful for determining user's interests and that the matchingwords would be suitable for AR objects. Additionally, the matching wordsin Seed provide the level information for each of the AR object thateither exist or need to be created. The level information is used tocreate links with other AR objects to form a chain that provides meaningand details matching the user characteristics with the products,advertisements, and services information that interest the user.

If the main content returns a no match with the Seed words, then inprocess AR4 text analytics and information extraction techniques areused to zero in on key pieces of information contained within. TextAnalytics involves lexical analysis to study word frequencydistributions, pattern recognition, tagging/annotation, data miningtechniques including link and association analysis, visualization, andpredictive analytics to turn text into data for analysis. InformationExtraction automatically extracts structured information fromunstructured and/or semi-structured machine-readable documents.Primarily this activity concerns processing human language texts bymeans of natural language processing. Commercial and open-sourcesoftware such as, IBM Languageware, Lexatytics, General Architecture forText Engineering (GATE), Apache UIMA, RapidMiner, and others areavailable to perform such functions.

Potential candidates of text analytics software have a broad collectionof data mining algorithms that can take some time to execute. Tospeed-up the computation and quickly converge on key words and phrasesthat match the usage classification of the system 100, process AR4 usestwo sub-processes. In the first sub-process, Attribute Repository Seed224 is used to see if any of the words or phrases matches the key wordsin the meta-data associated with the content or the web page on thescreen. If no match is found, then more computing intensive textanalytics software is used to provide new user characteristics or “closeto user interest” product and service information to create a new ARobject.

If the main content returns a match with the Seed words in step 320,then process AR5 is used to search the Attribute Repository database.The process of finding a match with parent and Child Objects wasdescribed earlier for FIG. 2 and in the Learn Mode.

It is possible that a match from process AR5 is not found in 330. If so,process AR6 creates a new Attribute Repository objects as describedearlier in the Learn Mode.

If in 330 a match is found, then process AR7 updates the objects in theAttribute Repository Structures 220. As described earlier in the LearnMode for FIG. 2, the update may include adding new Child Objects to thealready existing Parent Object.

Once the process is completed, the system loops back to process AR2 tobegin analysis of the next content in the cache.

Searching

A user can initiate search while watching content on the TV orexplicitly through the user interface because he or she wants to findsome information.

In either case, the search and the user actions are used by theApplication 212 (FIG. 2) to not only provide user interest-centricresults, but also to learn new “things” about the user.

The Application performs different kinds of searches in response to auser-initiated search. They are (1) “Internet search” where the resultsare returned from the World Wide Web using search engines, (2) a“results search” which involves searching through the results returnedby a search engine(s), and (3) a “local search” which involves searchingin the user domain when the user domain consists of hard drives, mediaservers, other storage devices, and subscribed cloud(s).

In one embodiment, when the user initiates a search while watching TV,the search “corresponds” to the category that is sourcing the contentbeing watched. Examples of this are searching for at TV show whilewatching TV, searching for music while listening to music, or lookingfor an item while shopping. The UI categories in these three cases areTV, Music, and Shopping, respectively. A search thus initiated wouldhave a direct relationship with the user's Attribute Repository objectsthat correspond to one of these categories.

FIG. 4 is a flowchart illustrating the Search Flow Diagram for contenton the screen. The illustration shows various processes, including userinput and display processes, that Application 212 performs usingcomponents in the Attribute Repository Structures 220. These processesenable the system to learn, perform searches, and present selectedresults to the user. The flow diagram is divided into three columns 402,404, and 406 with labels User, Display, and System, respectively,separated by two dashed lines labeled Time. The dashed lines between thecolumns depict flow of action or start of a process as shown by arrowsfrom one column to the other after a very short time or when the usertakes action. Each block in the flow has a label that is the action orthe process name.

The flow begins with process UI with a typical action a user may performon a TV. In process UI, a user turns on the TV and then switches towatch some content. The process D1 displays the content user has tunedto on the display screen.

With the content on the screen, the Application 212 begins its processof analyzing meta-data or content on the web page for keywords usingprocess S1. Meta-data is additional information that is typicallyprovided with content stream. If there is content on the web page, keysubject matter is isolated from other information, such asadvertisements, for analysis.

The analysis consists of finding words or phrases in the meta-data orthe content on the screen to determine if there is some knowledge in thesystem that could indicate a recorded user interest and other relatedinformation that can be used to make a decision on procuring additionalcontent or information that is in-line or similar to the content on thescreen and which could have a high user interest. Such analysis preparesthe system to present the most beneficial information to the user at therequested or an appropriate time when the user is most likely to take afavorable action.

The analysis is done using a combination of steps. The speed at whichthe analysis is done impacts how the user experiences the TV viewing.

Process S1 determines if the system has knowledge about the informationon the screen or does the system 100 needs to learn. There are threesub-processes to determine this. They are (1) use keywords in AR objectsstored in AR Structures 220, (2) use keywords stored in in the Seed 224in AR Structures 220, and (3) use text analytics tools. In the firstsub-process, process S1 uses the keywords stored in the AttributeRepository database and looks for matches in the meta-data or content onthe screen. These matches indicate that there is recorded information inthe system that matches or is similar to what the user is currentlywatching. In the second sub-process, the system compares words orphrases in the Seed 224 with meta-data or the content on the screen. Thematched words generate a set of new keywords indicating that themeta-data or the content on the screen is new subject matter that is ofinterest to the user and matches the end usage of the product. In thethird sub-process, the system uses text analytics and informationextraction tools and the Seed 224 to extract words and phrases frommeta-data and content on the screen to come up with words and phrasesthat are close or similar to the words and phrases in the Seed. This setof words would indicate a new interest by the user and the system willbe learning the interests.

In one embodiment, process S1 might only run sub-processes until a firstmatch, such as a matching AR object, is found and then go to the nextprocess with one set of keywords. This might be fastest for the system100 but it would delay the learning by system 100 of other informationof interest to the user that may be in the content and which the systemmight have learned if it executed sub-process two (match in the Seed)and sub-process three (text analytics) on the content on the screen. Inanother embodiment, process S1 might run all three sub-processes toobtain one set of keywords per sub-process as explained above and thengo to the next process where it begins the learning process based on thekeywords found using all three sub-processes.

Process S2 uses the up to three sets of keywords generated in process S1as follows. The second and third set of keywords created usingsub-process 2 and 3 in process S1 indicates that Attribute Repositoryobjects matching these keywords do not currently exist in the system.Thus, the system will be learning and needs to create AttributeRepository objects. The flow moves to the decision point 410 were “matchfound” is false. The flow then goes to entry point AR-A in FIG. 3 andcontrol passes to process AR1 in FIG. 2 as described above.

If the first set of keywords in process S2 is a match, this indicates anexistence of AR objects. In decision point 410, the “match found” istrue and so the control goes to process S3.

The set of matching keywords is used to locate the AR objects and theother AR objects that are linked to them. Note that each AR object wouldcontain keywords and other information as previously described. Thelevels in the Seed 224 as described before define how the AR objects arerelated to each. These AR objects, linked through forward and backwardlinks, form a chain. The chain defines a set of AR objects linkedtogether, that defines the interest of the user (highest level), to themost specific information on a subject (lowest level), and provides the“connection” between the user interests and the specific content,products or services. As previously described, the matched AR objectscould also have multiple links pointing to other AR objects that leadsto other termination points at the upper level and the lower level. Eachof these sets of links forms a new chain in Attribute RepositoryStructures 220 in FIG. 2. The collection of these chains is theinformation system has learned about the user and the products andservices that interest him or her. Each of these chains has a set ofassociated AR objects from where stored keywords are then extracted.

In process S3, the weight for each of the chain is computed. Weights areassigned to each AR object based on different criteria. The simplestcriterion is the “visit count”. Each AR object visit count isincremented by one, each time the AR object is included in a chain.Visit count alone may not be practical to suitably prioritize a user'spreferred or interest based information. To provide such priority and amore efficient weighting process, persistent or “sticky” usercharacteristics are assigned sticky weights that do not change withvisit counts. Examples of AR objects with sticky characteristics arecontent owned by the user, subscriptions or memberships to web sites,preferred websites where user has credit card information or web portalsthat are prioritized per agreements with the provider, merchant socialmedia, or advertisers. Sticky weights have values higher than the rangeprovided by the visit count. A higher preferred characteristic isprovided a higher sticky weight value while a lower preferredcharacteristic is provided a lower sticky weight value. Note that allsticky weight values are greater than the maximum visit count to ensurethat the computed weight of chains with sticky AR objects are higherthan one with just the visit count. Sticky weight values are providedthrough the Seed and with each word defining the characteristics. Theydo not change except when explicitly modified by an explicit command.

Chain weights might be computed by simply adding the weights of each ARobject in the chain, or in a more complex way by conditionally comparingthe weights of each AR objects such that the most preferred chainreceives the highest weight. By providing the sticky weights asdescribed above and using the visit count from all other AR objects inthe chain, a simple sum of all the weights of the AR objects in thechain is speedy and provides appropriate weights to the chains for thenext algorithm.

In process S4 the keywords from each chain is extracted to form sets ofkeywords, one per chain. Using each set of keywords, the systeminitiates searches, one per set, for content on user's hard disks, otherdevices, and the “cloud”, searches through preferred service providersor websites to locate content, products, services or advertisements, andthrough the Internet using commercial search engines such as Google orBing. A more detailed description of various searches described here isprovided later in association with FIG. 6. The system collects resultsfor each of the search initiated until it receives the last result forthe search corresponding to the last set of keywords.

In process S5 the group of search results obtained in process S4 isranked using weights calculated for each chain in the process S3. Thechain with the highest weight will have its group of search resultslisted first (ranked highest). This will be followed by the searchresult of the chain with the second highest weight, and so on.

Typical Internet search results from commercial search engines areranked based on the algorithms and criteria known to the search enginesproviders. These results do not match the deep and private user interestknowledge system 100 has gained and is owned by the user. Presenting allthe groups of search results accumulated in process S4 to the user on aTV or a personal device screen would be overwhelming and might reducethe user experience. Thus, in process S5 the results are filtered basedon criteria such as highest traffic rating (e.g., such as an AlexaTraffic Rank rating provided by Alexa.com, an Internet traffic rankingservice) or simply picking the first five (as an example) results forconvenience so that all results can be presented to the user in a moreconcise way. Traffic ratings or other such criteria ensure that popularand most relevant to end product usage results are displayed, whereasconvenience reduces computation and simply displays a predeterminednumber of top results. All filtered out results can be displayed on thescreen following the group of ranked results should the user decide toscroll pages and pages of them.

After filtering the results within each group of search results, thegroups are ranked in a list, beginning with the group that correspondsto the highest weighted chain. Thus, if the number of results in eachgroup is limited to five (i.e., n=5), the user will see up to fiveresults from the highest weighted chain, followed by up to five resultsfrom the next highest weighted chain, and so on. The rankings of resultsin such a manner ensures that the display will quickly show usersresults system 100 has found that corresponds to interests and theextensions to those interest and related to content, products andservices.

In step 450, if the user is watching a live or a streaming content onthe display screen, then control passes to process S6, during which asuitable time is determined regarding when to display the informationthe system 100 has gathered related to the content being watched andmatches the interest of the user, and then control passes to process D2.

Process D2 is invoked when the system detects that a streaming or a liveadvertisement is about to start. Such detection is identifying asuitable time when the system can insert its own content. If suchdetection can be made, the system prepares to play the interest-centricadvertisements out of the information it has gathered in process S5.

Process D3 is invoked following process D2 if the system detectsplayback of advertisements. The campaign-based advertisements beingbroadcasted or streamed live seldom match user interests and hence lackeffectiveness. Instead of such advertisements, the system will playinterest-centric advertisements from process S4 ensuring that they arehighly relevant to the user and thus might increase the probability thatthe user will take as favorable action on them. In addition to theadvertisements, a MORE button is displayed on the screen that the usercan point to and click to request more information.

If the user clicks the MORE button displayed on the screen in processU4, process D4 will list the ranked results accumulated in process S5.

Since the information on the screen has changed following the executionof process D4, the system starts the process all over again beginningwith process S1.

If, however, in process U3 it is determined that the user clicked the“Resume” button, content playback will resume when control passes backto process D1.

If the information being displayed on the screen is determined in step450 to not be live or streaming content, the system displays in processD5 the ranked results accumulated in process S5. The exit from processD5 is to process U5 where if it is detected that the user clicked on oneof the new information presented on the screen, the system starts theprocess all over again beginning at process S1.

The flow diagram 400 also shows a process U2 during which a user can“Pause” the content currently playing through a remote control. This isanother example of a “suitable time” when the system can play its owncontent. Advanced STBs provide a function where the currently playinglive or streaming content is continuously recorded so that if they arepaused anyplace, they can be “Resumed” without missing anything.Assuming such a feature exists on the STB; the system will start playingthe interest centric advertisements in process D3.

In addition to displaying information to the user, the Application 212also begins the learning process as described above in the Learn mode.The process S5 has compiled and ranked the search results based on thekeywords in the chains of AR objects and their weights. As previouslydescribed, weights include sticky weights that puts AR object chainswith properties such as owned by the user, at as higher rank than theresults from other chains where the AR objects properties does notinclude sticky weights. The process S7 looks at the group of rankedresults from process S5 and focuses on the search results that do nothave the sticky weights. This group of search results is based on thekeywords provided by a chain of AR objects and is from the Internet.They exclude items that are already owned by the user (system 212 hasknowledge about those items). Process S7 then fetches the webpagepointed to by the search result. The control then goes to entry pointAR-A (to step 310) in FIG. 3 to begin the system 100 learning process.

A user can also initiate a search through a “Search” category of the UI,which is a search for anything. FIG. 5 shows the various process steps500 the Application 212 goes through in servicing such a search and howthe system learns about the user's characteristics. Referring to FIG. 2,the User Initiated Search flow begins with an input received via aremote control over flow-link 204. The Internet search request andresults are communicated over flow-link 208, and the search results aredisplayed through flow-link 210.

In FIG. 5, the flow begins with a user-initiated search in step 502. Acheck is made in step 504 to determine if the search was initiated fromany UI main-category except Search. If it was initiated while in a UImain-category, then the search can be refined with the category. In thiscase, process S21 is invoked. Otherwise, this is a request for directsearch and process S11 is invoked.

Process S11 is the first path when the user is looking for “any”information. Keywords and phrase would be extracted from the searchwords user entered. The process limits the scope of analysis to thephrase or words exactly as entered by the user. The system then goes toentry point CS-A (to step 414) in the Search Flow 400 of FIG. 4.

If the search was initiated from within the UI main-category, processS21 is invoked. This process appends the UI category information torefine the search. As an example, if the search was initiated fromwithin the UI main-category of Movies, the word Movies would be added asanother search term to limit the results to “movies”. Similarly,Shopping UI category can be added to limit the search if the search wasinitiated from within the “Shopping” category. The analysis and the restof the process will be followed through entry point CS-A (to step 414)in the Search Flow of FIG. 4.

Process 600 in FIG. 6 expands on the search part of the process S4 inFIG, 4. As noted earlier, the system 100 conducts searches for contentin the user domain, content on the websites through the Internet whereuser has accounts, and lastly, the entire Internet. The mechanisms forsearch are different in each of the three cases and are supported by therespective AR objects.

Assuming that content in user's domain has the chain with the highestweight, the AR object matching the user content would contain a path tothe catalog director or a file name that contains the list of user ownedcontent and the path information. Path information consists of thedirectory and file names where the content is located on the hard drive,private networked path or mapping to other user owned devices, and theInternet address (e.g., URL, IP address, etc.) of devices that can beaccessed over public network or through the Internet. In process S31 theAR object for user owned content is located and the device paths for alldevices where content may be located is extracted. Next, in process S32a search over each path (each device) is initiated to determine if thereis a match between the primary keywords and the words in the contentcatalog. Primary keywords are extracted from AR objects in the chainthat are tagged with key properties of the content. Examples of keyproperties of the content are title, artist name, etc. Primary keywordshelp in locating content where full set of keywords that also maycontain user characteristics and other user centric information may notreturn a match. Until all the user's devices (e.g., storage 106) aresearched, process S32 is repeated. Then control moves from step 606 tothe next process S33.

Process S33 determines if the AR object in the chain show that useraccounts exists. If accounts exist, the process locates the website URLsand the sign-in information. Process S34 then connects to the websiteusing the URL and the Application 212 signs in using stored sign-ininformation. Next, the process locates the “Search” entry region of thewebsite and enters the primary keywords in the search field. A search isthen initiated. The result returned will contain primary information oncontent that are offered through the website and where user has anaccount. The process then records the URL along with all search resultitems returned in the chain's result group.

Step 612 forces the system 100 to repeat process S34 until content fromall websites where user has accounts has been located. Then, in processS35, the next chain and the set of corresponding keywords is located.

Process S36 initiates an Internet search with the full set of keywordsdetermined for each chain by process S4 in FIG. 4. The results returnedare recorded in the search result areas of the chain whose keywords wereused to do the search. If searches were not initiated for all thechains, the control goes to 618 and process S35 to perform the nextsearch. If searches were initiated and all the results received, thecontrol passes back to process S4 in FIG. 4.

An illustration will serve to better show what the user experiences: Inthis example the user is in the UI Movies category and searches for“Famous Artist 1”. Using the user's Attribute Repository for Movies, thematch finds “Famous Artist 1” and the path information for movies thatmay exist in the user's domain (hard drive, media server, other devicesand the cloud) in one of the chains. Local content has the highestweight. Next, a restricted Internet search is done on websites where theuser has accounts by automatically logging in (the log-in informationwas also retrieved through a match with Attribute Repository object inthe next lower weighted chain). Lastly, the Application 212 willinitiate a general Internet search with complete set of key words fromthe chain that better reflect the user's characteristics and the linkedcontent, products, advertisements and services. The results returned arebetter defined and have a high likelihood of matching the user'sinterests or product usage.

The order of the search results presented to the user would be: (1) userowned movies with Famous Artist 1, (2) Websites where user has anaccount that offers movies with Famous Artist 1 as a star, (3) othersites offering Famous Artist 1 movies, DVDs with best deals listed first(with price set as a priority), (4) new movies Famous Artist 1 is in andtrailers or description. Then, if the Attribute Repository has ChildObjects in other chains for Famous Artist 1 (indicating other userinterests) in fashions, events and gossips, the search result list wouldshow, in rank order, (5) Famous Artists 1's fashions, (6) events withFamous Artist 1, (7) gossip about Famous Artist 1 and so on.

Note that in describing the Learn Mode example of a movie with FamousArtist 1, the system learned about the artist's fashions and othercommercial items and created Attribute Repository objects. In theexample above, the links to these Child Objects would also be recognizedby the system and the links to the websites of the merchants or productswould be listed in the search result. If the user clicks on the pointer,that merchant or the product's website with specific product or modelwill be displayed.

Alternately, if the UI main-category was Shopping and the user enteredFamous Artist 1 name in the search, the merchant and product relatedchains with AR object linked to the artist would be displayed first.

Presentation Repository Structures Algorithms

As previously described under Convey Mode, the display of the UserInterface with its dynamic nature is managed based on the PresentationRepository Structures 250 shown in FIG. 2.

In the example of the display of the UI, main-category, such as TV,movies, or shopping is displayed as a group to make a coarse selection.If one of these categories is selected, then fine selection such as thenames of the TV shows, names of the movies, or the names of shoppingestablishments is provided as a sub-category of the respectivemain-category for selecting the content. Navigation choices on the mainand sub-categories are presented if the user moves a cursor to one ofthe choices visible on the screen and selects additional actions usingan input device.

All of the structures in the UI are adaptive. Items can be added ordeleted in the main-category, sub-category or the Listings. Thiscapability is provided through the Presentation Repository structures250.

The main and sub-category as well as navigation choices might bedisplayed as buttons, icons, listings, or a combination of any or all ofthem. The appearance of the display is typically a matter of choice of abusiness or manufacturer. FIG. 7 is one exemplary UI representation 700of how the screen regions could be allocated for interfacing with theuser.

The buttons, icons, and listings have several parameters associated witheach of them. The parameters include the position and size of thebuttons, e.g., button 712 shown in FIG. 7 as it would appear on thescreen. Other parameters include the font of the text, a link topictures or content; generally these parameters are not shown on thescreen. The links typically point to a file name on local storage forspeed however they could be pointing to a URL or a TV channel if thedesign calls for such items to be displayed in the buttons.

The main-category region 710 of the UI screen 702 is represented by acollection of Presentation Repository objects shown as parent objects inFIG. 2. Upon selection of a main-category button, linked Child Objects(FIG. 2) are selected that are presented as sub-categories in region 730in FIG. 7. If a sub-category is selected and there are more choices,they are presented in the listings area 720.

The aesthetics of the user interface and how much information ispresented on a screen for a TV experience or a mobile device variessignificantly from screen to screen. Simple UI display, such as TV andTV shows, can be represented in the main and sub-category alone. AnInternetTV listing that offers chokes from around the world can beselected by region, genre, or language where the selection possibilitiesare provided in the sub-categories. The sub-categories further offerchoices. Here, each region, genre, or language can offer more choices,possibly creating a complex presentation challenge to fit on one screen.It is possible to put these choices in the listings area 720. Thelistings area or the sub-category region 730 can then offer more choicesthat can make the screen very crowded.

The disclosed embodiments provide a method where very complex UI screenpresentation challenges might be overcome in a way that preserves the TV“experience”, i.e., the look and feel of watching conventional TV. Thefamiliar screen and the presentation aesthetics are persevered and theuser can simply page through multiple screens. These methods, in one ormore embodiments, rely on Presentation Repository structures 250 in FIG.2 with parent and multiple-linked Child Objects shown as PR 252 andPCO1-1 . . . PCO2-1 . . . respectively.

Too many rows and columns of buttons on the screen might make the UIvery cluttered and provides a poor user experience. If the navigationcalls for additional button choices when the UI screen 702 is alreadydisplaying current button choices, the Application uses next level downPR Child Objects to maintain UI aesthetics by moving or shifting thesub-category buttons in 730 into the main-category region 710, anddisplaying the next level down sub-category buttons (from the ChildObjects) in the sub-category region 730. This action resembles a “shift”of previous sub-category region (level 1) to the main-category region,illustrated by arrow 740, and the shift-in of the next level down(level-2) sub-category region to where the previous sub-category was alevel-1 sub-category, illustrated by arrow 742. With linked child PRobjects, the aesthetics of the UI presentation 700 is maintained. Theprocess can continue until there is content selection or the use exits.A “back” button on the screen (not shown) or on the remote control wouldcause the Application 212 to “walk back” through each screen (ChildObject) to the main-category screen (home). A “home” button, ifprovided, would take the system back to the top level UI screen. FIG. 8shows an example of the sub-category level 1 in region 730 in FIG. 7shifted into the main-category region 810 with the cursor selecting thenow sub-category selection button 812, in FIG. 8, and a level-2 ofsub-category shifted into region 730 in FIG. 7 and appearing here inregion 830 in FIG. 8.

FIG. 9 is an exemplary flowchart 900 which shows how the User Interfaceactions, through a remote control or the equivalent, work with thePresentation Repository Structures 250 and its objects to provide the UIdisplay. To simplify the input device actions for illustrative purposes(such as from a remote control or a mouse), two generic action itemterms, (OK) and (Menu) are defined for purposes of this embodiment. (OK)is synonymous with moving the cursor to a button or an item on thescreen and clicking or explicitly pressing the OK button on a typicalremote control. This action causes the system to “select” the item where“select” means that the item is highlighted to offer more choices, or tostart playing content (such as a TV show or movie). The selection isrepresented by the Child Objects and in the simplest case might berepresented by the linked list of the Child Object PCO1-1, PCO1-2, andPCO1-3. As an example, if the main-category selected was TV, then PCO1-1would represent the first TV Channel, PCO1-2 would represent a second TVChannel, and so on. The parent or the child objects also have a secondlink that points to a second list of the child objects similar to theones shown in PR structures 250 in FIG. 2. For parent object PR2 in 252in FIG. 2, this set of Child Objects would be PCOM1-1, PCOM1-2, andPCOM1-3 (not shown in FIG. 2). They are at the same level as the ChildObjects PCO1-1, PCO1-2, and PCO1-3. These second set of Child Objectsprovides the Menu button choices. (Menu) means pointing the cursor to abutton or item on the screen and then pressing the (Menu) button on theremote control. This action causes the system to show additionalnavigation choices or selections.

The flow 900 begins execution with an action performed in step 902 usingan input device. Process UP1 in the Application 212 locates the parentobjects in the Presentation Repository Structures 250, shown as PR 252,and determines the physical characteristics of each button in the UImain-category. Any of the PR parent objects can be provided with thedefault property of appearing highlighted when selected. For the productusage of TV, the default is the TV button. It will appear highlightedwhen the UI screen first appears.

The Application 212 (FIG. 2) locates the linked Child Object, PCO1-1 inFIG. 2 that is linked to the parent object PR2 in 252 that matches thehighlighted button “TV” on the UI screen. It then finds the parametersfor each of the buttons in the Child Object list PCO1. This list PCO1 isthe sub-category that corresponds with the highlighted main-category.Again, any of the items in the sub-category can be highlighted. For TV,we chose the last TV channel played that the system remembers.

The Application then tunes to the selected channel and displays it onthe screen. At the same time, the UI screen is displayed overlaid on topof the playing TV channel. This is shown in FIG. 13 and discussed later.

With the UI screen displayed, the system waits for user input to eitherchange the channel or select a different main-category and, in analternative embodiment, before the UI display times out. Cursormovements are tracked. At UI display time-out, the UI screen vanishesfrom the display screen and the user only sees the content.

In process UP2, the cursor is tracked and the Application waits for theuser to select an item. With this tracking, the Application selects theparent object in the Presentation Repository, corresponding to UImain-category, or the linked Child Objects, corresponding to UIsub-category.

In step 904, if the user moves the cursor to a main-category button andpresses (OK) on the remote control, that main-category button will beselected. Process UPP3 selects the PR parent object corresponding to themain-category button, and using the display properties in the PR object,highlights that button. The links in the parent PR object is used tolocate the Child Objects. Using the Child Object list, sub-categorybuttons are then displayed in the sub-category region. As an example, ifthe user moves the cursor to the InternetTV button in the main-categoryregion and presses (OK), the InternetTV button will be selected(highlighted) and the sub-category buttons (names of the countries orregions) corresponding to the InternetTV main-category will be displayedin the sub-category region.

As previously discussed, for simplicity the remote control selectionchoices are defined as (OK) and (Menu). An action is taken if the cursoris pointing at a UI button corresponding to a Child Object such asPCO1-1 and the OK button is pressed. If the cursor points to a UI buttonand the Menu button is pressed, Child Object PCOM1-1 is used to displaythe menu choice buttons. This is true for the main-category and thesub-category UI buttons.

If the design calls for more choices, more Child Objects-would be linkedto the PR object reflecting multiple choices.

In step 904, if the user presses the (Menu) button on the remote controlwhen the cursor is pointing at a main-category button, then process UPP4is selected. On pressing the (Menu) button, the second Child Object listPCOM1-1 is selected. If the user moves the cursor to one of the menucommands, and presses (OK) on the remote control the Application will goto the matching Child Object, e.g. PCOM1-4, and take actioncorresponding to that button. Examples of main-category command buttonsare: move the button up, move the button down, delete, change color,etc.

Process UPP3 and process UPP4 both exit to the beginning of process UP2to reflect that the system is tracking the cursor to determine the nextuser action.

If after changing the main-category selection the cursor moves to thesub-category region, the user has similar action choices as he or shehad for the main-category. The user can select an item by pressing theOK button or bring up the menu for commands by pressing the Menu button.The decision point “Sub-Category Actions” 910 in FIG. 9 illustratesthese choices.

If, as determined in step 910, the user points the cursor to asub-category item on the UI and presses the (OK) button on the remote,process UPC3 will be selected. In this process, a linked PR Child Objectis used to determine if the cursor points to content (such as a URL orpath) or if there are additional choices or information that needs to bedisplayed. If the Child Object indicates content, it starts playing. Ifthe Child Object points to additional information or choices, then theprocess moves to the next decision point. The main and sub-categoryitems are highlighted.

After process UPC3, step 920 labeled “content, listing or sub-category”reflects possibilities with content or other choices available thatprovides more details. If the sub-category button points to content andthe user presses the Menu button on the remote control, process UPC4will check for content and pull up command choices for display. Thecommand choices examples are: user wants to tag this content as“favorite”, user wants to record the content, delete the content, or anyother command the design and features call for. One of the commandbuttons displayed is Play, which if clicked (by pressing OK), will causethe content to begin playing. The control then goes to process UPC5.

In process UPC5, if the user presses the Exit button on the remotecontrol, the system 212 will remove the UI display and go back to thestate where content that was being played when the user entered the UIwould be displayed. If the user does not do anything, then after atime-out elapsed time, the system will go back to the content beingplayed at the time user entered the UI. The time-out time is set up inthe system 212 to provide the time UI screen stays overlaid on thecontent without any actions by the user.

In step 920, if the user presses the OK button on the remote control,the control goes to step 924 where the button is checked for link tocontent. If the button points to the content, it begins to play throughprocess UPCC3. The control then goes to process UPC5. The actions inthis process have already been described above.

In step 924, if the button does not point to content, then in thisexemplary embodiment there are two possible ways additional informationcan be listed on the display 122. Step 928 resolves this. If theadditional information is in a sub-category, control goes front step 928to process UPCC6. This process manages level-2 subcategories. First, theprocess UPCC6 moves the level-1 subcategory to the main-category region810 in FIG. 8. Next, the process locates the next level PR child objectin PR Structures 250 in FIG. 2 that represent level-2 sub-category andmoves them to the level-1 subcategory region 830 in FIG. 8. The controlthen goes to process UP2 where it tracks cursor's movement, user actionor time-out. Note that the main-category region 810 has level-1sub-category in it.

If step 928 determines that the additional information is a listing, thecontrol goes to process UPCC5. In this process, the PR child objectwould indicate if the listing consists of a path to a directory or afile on a system, a URL, or some other object that has enoughinformation that it can be displayed on the display 122. Once thelisting information is located, the Application 212 fetches the listingcontent and displays it in region 820 of FIG. 8. The listings can benested; meaning an item in one set of listing may in turn point toanother listing. In such scenario and in one embodiment, the maincategory 810 and sub-category in 830 in FIG. 8 remains on the screenwhile the listing keeps changing. At some point, when the listing pointsto the content and the user clicks it, the content begins to play. Thecontrol then goes to process UPC5. It is easy to see that linking ChildObjects to a PR parent object where the Child Object also contains UIdisplay properties, one can create various combinations ofmain-category, sub-categories, and listings. The same techniques canalso be used to create many other UI displays with various combinationsof regions, buttons, menu options and UI navigation.

If, in step 910, the user points the cursor to a sub-category item andpresses the (Menu) button on the remote control, the Child Object linkedto the sub-category item and corresponding to the Menu button will beselected. This Child Object is similar to PCOM1-1, PCOM1-2 describedabove for selecting menu choices in the main-category. The control goesto the process UPCC4.

The process UPCC4 uses the Child Object list to present the “Menu”button choices that corresponds to this sub-category item. As anexample, if the sub-category item is a TV show, and the user has pressedthe Menu button, menu button choices will be displayed that may show,buttons such as: “Add to Favorite”, “Delete from Favorite”, Record, or“Back” as the menu button choices. These choices correlate with oneChild Object each, with a particular button represented by a ChildObject PCOMx-y where x and y represent the x'th item in the sub-categoryand y'th command in the menu choices. Track the cursor and wait for theuser to point to one of the command and click OK. When the user selectsthe command, the Child Object corresponding with the command will beselected. This Child Object provides the path to the process inApplication 212 that can carry out this command. Following the executionof the command, the control goes to process UP2 where it waits for thenext action by the user.

It is possible that sub-category item commands may not executeeverything needed with just one menu command because more selections areavailable and other choices can be made. The PR Child Object structurefor the sub-category and the next level linked Child Objects PCOM2-1,PCOM2-2, . . . provides the mechanism through which second level menubuttons might be presented. In one embodiment, the menu options mayappear on the screen one on top of the other, with the newest one fullyvisible, to provide a mechanism to the user to remember the content onwhich the action is being performed through main and sub categories andto walk back using the “Back” button.

FIG. 10 is an exemplary flowchart illustrating, in one embodiment, themenu buttons and the actions performed by the system 100 in the processUPCC4 in FIG. 9. The flow 1000 begins with process UPCC4 where the useris presented with, in this embodiment, three choice buttons by pressingthe Menu button on the remote control at step 910.

For illustrative purposes, the menu choices include an “Explore” button,a View button, and a Record button. As previously stated, other buttons,such as Add to Favorites, Back, etc. might be provided.

The “Explore” button is similar to the command in various operatingsystems that lists libraries, directories, and files. Step 1010 showstracking of the remote control pointer and whether the user clicked onit while it was pointing at the Explore button in the menu. If thebutton was clicked, the control goes to process PRUIE1. This processenables the system 100 to access the storage 106, and storage on otheruser owned networked devices and the cloud through the network port 102.The Explore Child Object process PRUIE1 extracts the pointers to thecatalog system 100 maintains on storage 106 for content locally stored.Similarly, using the path information to other networked devices, itaccesses the catalogs maintained on those devices. Next, process PRUIE2compiles the list of all contents list accessed from the catalogs andpresents it to the user in the listing area 720 in FIG. 7. The exit fromprocess PRUIE2 is back to process UP2 in FIG. 9 where the system 100waits for the next action by the user.

Similar to the Explore button, if the user points to the View button onthe menu and clicks it, step 1020 will pass the control to processPRUIV1. In this process, the content pointed to by the child object isplayed. Process PRUIV2 causes this content to be displayed on monitor122. The exit from this process is back to process UP2 in FIG. 9 wherethe system 100 waits for the next action by the user.

Lastly, FIG. 10 shows the Record button that, if pointed to and clickedby the user, goes through step 1030 to process PRUIR1. In this process,the system initiates the process to begin recording the content on thestorage 106. Generally, a record function also provides options for, forexample, beginning recording immediately or at a set time. Optionally,the recording can be aligned with the electronic programming guide (EPG)to record a specific program at a given time. All such choices forselection and input can be presented through more menu buttons usinglevel down linked Child Objects. When the recording is set up, processPRUIR2 sets the system 106 recording schedule library by modifying it.Recording Schedule Library is a list of all content or programs that arescheduled to be recorded by the system 100 at the “recording begintime”. Such library is also useful to enforce recording rules that mayinclude limits on the number of tuners that can receive the programs formultiple program recording, digital rights management, recordingconflict, no space available, etc. The exit from this process is back toprocess UP2 in FIG. 9 where the system 100 waits for the next action bythe user.

Presentation Repository Update Flow

Presentation Repository update is performed to reflect new changes inthe Attribute Repository Structures 220 that results from system 100learning new user characteristics, or new information on content,products, advertisements or services. The scope of machine learningexecuted by the system 100 is aligned with the product usageclassification. In one implementation, the product is used forentertainment, commerce, social media, and advertisements. Based on theproduct usage, the Attribute Repository Structures 220 include usercharacteristics and information on movies, clothing, transportation,food, music, TV shows, videos, people, places, etc. in the parent andchild AR objects. Benefits to the user, service providers, merchant thatoffer products or services and the advertisers can only be realized ifthe things the system has learned (machine learning) can be displayed tothe user automatically. This is done through the PresentationRepository.

The User Interface main-category choices can be aligned with the productusage classification. The UI screen presentation and layout might haverequirements from the service provider, such as subscription choices andcontent or product promotion priorities. An example of subscriptionchoices is “limited content accessibility” provided by the serviceprovider that cannot be changed by the user. Such limits are providedto, for example, protect children if the distributed content mightinclude mature or adult material. Content of product promotionpriorities might be employed when the service provider requires thatcontent through their distribution channel or products from theirpreferred merchants are displayed first. In one example, the UImain-category items for the first product usage classifications are TV,InternetTV, Music, Radio, Movies, Shows, Videos, Shopping, Social Media,Search, and so on. These items can be prioritized or restricted, basedon the requirements or business criteria mentioned above.

The processes described up to this point show how the AttributeRepositories Structures 220 and the Presentation Repository Structures250 in FIG. 2 are created and used in the system 100. When the user isviewing content, changes in the AR Structures 220 can occur as fast assystem can learn. At the same time, PR Structures 250 needs to reflectthe changes in the AR objects as quickly as possible to show the latestupdates. If the system 100 has significant computing power, this can bedone with ease. In the embodiment where the computing power is morelimited, the update is done when the computing requirements are low suchas when the device is not in use.

The flow uses tags, called UI category item tags that are saved in theAttribute Repository objects and were created as previously described inthe Learn Mode. The Seed 224 keyword that created the AttributeRepository object also has a parameter indicating if the AR object wouldbe part of the main-category, sub-category, or no category in the UI.This UI category tag is saved in the AR object. When the system linksthe AR objects, the linking forms the chain that would contain an ARobject with main-category tag and other AR objects with sub-categorytags. The chains in the Attribute Repository describe the relationshipsuser characteristics have to the product or services. The UI categorytags describe the relationship AR objects have in the UI with respect toeach other. It is not necessary that the UI relationship in the AR chainbe exact. The PR update flow can sort the AR objects and can create PRlinking to reflect the accurate relationship of these AR objects in theUI. As an example, if an AR object is created with a “Movie” UImain-category tag, AR object in the chain with keyword “Movie Title”will have a subcategory tag. Other AR objects in this chain or otherchains matching the user characteristics will not be made part of the UImain-category Movies. If the system is configured to learn new UImain-category items, then these main-categories are also added to theSeed so that association can be made between the new AR objects and thenew PR objects.

An exemplary Presentation Repository update flow 1100 is shown in FIG.11. A Presentation Repository update trigger, winch is a timed processthat triggers the Presentation Repository update process, is used tostart the update. Process PRU1 stops all Learn-Mode activities for theAttribute Repository 222 and the Child Objects. By stopping theLearn-Mode activities the possibility of the PR update processcontinuously chasing the AR Learn process is eliminated. The cachingactivity continues per process AR3 in FIG. 3 to capture user activity,if any, during this time.

In process PRU2 the first Presentation Repository object thatcorresponds to a UI main-category is located and the PR updating isbegun. The first UI main-category item is the one that appearshighlighted when the UI first appears on the screen, for example, TV.

In process PRU3 Attribute Repository structures 220 is queried to findany AR object that contains the UI main-category tag and is the oneabout to be updated in the Presentation Repository. In one embodiment,an “updated” flag is added to the updated AR object. If the AR object'supdated flag is set, it means that new AR objects were added and thismain-category item needs to be updated. If the flag is not set, thesystem can skip this update. If the AR object reflects an update,corresponding PR objects and the links (i.e. the Child Objects) will beupdated.

Then process PRU4 begins the update process. Earlier it was discussedthat a Presentation Repository parent object represents a UImain-category item. The PR parent object is linked to PR Child Objectsthat represent UI sub-categories. A PR Child Object is linked to thenext lower level Child Object that represents a UI list (FIG. 2, FIG. 7and FIG. 8). Thus, the UI display chain consists of (in order): UImain-category item→UI sub-categories display→UI list.

With the match, an Attribute Repository object has been located thatmatches with the UI main-category item and the Presentation Repositoryparent object. Referring to FIG. 2, the PR Template 256 in PresentationRepository Structures 250 defines what can be linked to the PR parentobject and the properties such items may possess, such as the type ofcontent in the UI sub-category. The content type could be a title, suchas title of the movie in the UI main-category of Movies, or a country inthe UI main-category of InternetTV.

In one exemplary embodiment, the PR update process in process PRU4 issetup to meet three objectives. (1) The information needs to bepresented to the user per the UI guidelines and aesthetics. (2) The UIpresentation needs to be dynamic in nature to reflect user tastes andpreferences that may have changed—this means items can be added, deletedor presented in different order. And (3), each UI item must have fastaccess to other information system has learned about the user so that atthis instant in time when the user is watching something that interestshim or her, other related user interest centric information can be usedto predict and offer content, merchandise or advertisement type ofinformation for greater effectiveness and most benefit to the user.

The update process for each UI main-category item is iterative innature. The number of iterations is based on the number ofsub-categories or levels illustrated in the PR template. If a PRtemplate for a main-category, such as movies, shows one Child Objectwith keyword as “movie title”, then the sub-category level is one andonly titles will be listed in the buttons in the sub-category region ofFIG. 7. Alternately, if the PR template illustrates two PR Child Objectslinked to the main-category of movies, then there will be two levels ofsub-categories. The first PR Child Object in the template with thekeywords “movie genre” would indicate that level-1 sub-category 730 inFIG. 7 has buttons that provide the genre for the main-category ofmovies. The second PR Child Object in the template with the keywords“movie title” would indicate that if any of the movie genre buttons wasclicked on the UI screen; the level-1 sub-category would move to themain-category region 810 in FIG. 8 and a second level sub-category wouldappear in the region 830 with buttons where each button is a movie titleavailable within that genre. An exemplary showing of buttons in the mainand sub-category is shows in FIG. 13 and is discussed in more detailbelow.

The process PRU4 first determines the number of sub-category levels fromthe PR template matching the Attribute Repository object identified inprocess PRU3. Let us assume this umber is Lx-max. Next, a loop is setupthat will update various sub-categories up to Lx-max level as describedbelow.

The loop begins by searching for all Attribute Repository objects thatmatch the keywords or phrase in the PR Template Child Object forLevel-1. As an example, the Level-1 PR template for main-category ofmovies would provide keywords such as “movie title”, if movie titlesonly were to be included in the Level-1 sub-category. Alternately, thekeywords would be “movie genre”, if movie genre was to be included inthe Level-1 sub-category. Any AR object found with such keywords wouldprovide the necessary information or pointers to be included in the UIbuttons.

Next, we locate the “visit count” for each matching AR object. It wasdiscussed earlier in the document that AR object visit count isincremented by one anytime the AR object is included in a chain forsearching information to be presented to the user. Also recall that thechain is created following forward and backward links in AR objects andis a mechanism for matching a user characteristic with a content,product, brand name, or advertisement. If a matched AR object is linkedto many chains, it would have a high visit count. A high visit countimplies that this AR object either directly effects the linking of theuser characteristics with information about product and services or is asignificant contributor to it. Such an AR object would be a greatpredictor for related content, product, and services and would implymuch more than simply labeling this content as favorite. The visit countis used to sort the matched AR objects in a descending order of thevisit count. The first UI button displayed to the user is based on thehighest visit count AR object and would reflect the “most beneficial”entry to the user.

Next, we create a linked list of PR Child Objects in the PresentationRepository. These Child Objects are linked to the PR parent objectrepresenting the UI main-category. Each entry in the link list iscreated in the same order as the AR object sorted list. Each entry usesthe PR Template 256 to fill in the properties, links, navigationchoices, and other essentials from the AR object for the display buttonin the UI. Note that the Child Objects would include objects such asPCO1-1, PCO2-1 as shown in PR Structures 250 in FIG. 2 and synonymouswith the action of user clicking the OK button on the remote control. Asecond Child Object list would also be linked, consistent with the ChildObject list for Menu button options as described before. Thus, for eachPCO1-1 Child Object that will be followed when the user presses the OKbutton on the remote control, PCOM1-1 Child Object (not shown) isprovided that will be followed when the user presses the Menu button onthe remote control. Once the PR Child Objects are all linked, the systemwould have completed creating the sub-category level that started withthe loop.

The next step in process PRU4 is to increment the sub-category level andcheck it against the maximum number of levels in the PR template. If thePR Template indicates that there are more levels, the system repeats theloop; this time for each PR Child Object in the Level-1 sub-category,until all Level-1 PR Child Objects have received the Level-2sub-category linked and sorted PR Child Objects. In other words, Level-1subcategory buttons in the UI would receive the UI menu choices andnavigation buttons for Level-2. The process repeats itself until thereare no more levels left in the UI sub-categories and the maxsub-category level has been reached.

It is possible that the information available at some level of UIsub-category is such that it cannot be easily put in buttons. In suchevent, a list can be compiled that can be displayed in the listing areaof FIG. 7 or FIG. 8. Such lists are created from chains after matched ARobjects are found. In one embodiment, the list can have headings forreadability or to show association with the sub-category buttons on thescreen. Under each heading, information extracted from the chain withmatched AR object is inserted. At the end of this process, the listcomprises of all the information that matches the user's interests andis related to the one item in the UI subcategory. Once the listing iscomplete as a file, the pointer to the file will be added in thePresentation Repository Child Object for the sub-category. When the userclicks the sub-category button on the UI, the listing area of the UIdisplay will then show the content of the file.

After all new and updated Attribute Repository objects are processed forone UI main-category, the Application goes to the next UI main-categoryitem in process PRU5. If, as determined in step 1102, this is not theend of the UI main-category item list, process PRU2 is entered where theprocess of updating the UI sub-category and list is repeated again forthe new UI main-category item.

Once all the UI main-category items are processed, the Application 212executes process PRU6 that restarts the Attribute Repository updateactivity that was paused at the beginning of this process. TheApplication then terminates with this flow.

User Interface

The User Interface (UI) is the graphics and fonts that are displayed onthe screen overlaid on top of the currently playing content or a blankscreen.

The graphics include buttons, navigation options, layout, color, fonts,etc. that can be shown on the screen. As discussed earlier, objects inPresentation Repository drive each element of the UI. The presentationrepository object describes how a particular entity of the UI, such asbutton, menu, or listing is displayed on the screen. Display propertiesin the PR object describe the appearance. This includes the itemlocation on the screen, border, color, font, embedded picture or video,etc.

A Presentation Repository object can also indicate that the content isto be displayed in full screen or a window. A window is beneficial iffor example, a user is watching a show while simultaneously interactingwith the show site, a social site or simply with friends over InstantMessaging. In this case, both windows will be shown. In one embodiment,the UI overlaid is scaled to full screen.

FIG. 12 shows an exemplary flow of UI actions, various processes inApplication 212, and menu operations to create a new user. The system100 learns about a user characteristics and matches them to content,products, services, advertisements, etc. that are needed for systemusage classification. Since the information learned by the system 100about the user is deep and might be highly personal, other possibleusers will need to have their own set of data in the AttributeRepository Structures 220 and the Presentation Repository Structures 250in FIG. 2. FIG. 12 illustrates these Structures collectively for threeusers as 1220, and specifically as Structure set 1220-1 for the userFamily, Structure set 1220-2 for the user User 2, and Structure set1220-3 for the user User 3. A product such as a STB or a gateway withsufficient storage 106 could maintain structures 1220. Other productsowned by the user that are not used by others, such as a mobile phone,will only need to have one set of structures 220 and 250. As discussedearlier, in one embodiment where other user owned devices like themobile phone that have limited storage 106, have a limited number of ARobjects in AR structure 220 and PR objects in PR structure 250 toconserve storage space. A method discussed later provides the processthrough which a mobile phone like device can “synchronize” with a STB ora gateway type device to maintain most up to date objects in structures220 and 250.

FIG. 12 flow 1200 begins with step 1210 when the user turns on the TV.In one embodiment, Application 212 begins the process Startup where thesystem 100 defaults to a user profile named “Family”. This profilecontains AR objects in structure 220 that the system 100 learned fromthe user named Family, by accessing entertainment content or web pagesthat the “Family” was careful about in selecting. The structures usedare shown in 1220 as the set 1220-1. In another embodiment, the profilefor Family can be restricted by the service provider to have a “limitedaccessibility content” profile as a default profile as describedearlier. Next, the process Startup selects the initial program todisplay. For TV usage classification, the program is typically selectedfrom the last channel the user had tuned through the tuner 112. Afterselecting the content, process Startup initializes the processes andlocates the parent PR object matching the TV main-category. Using theChild Objects linked to the parent PR object, process Startup preparesthe content for display and prepares the UI that will be overlaid on it.In step 1214, the content prepared for display (last TV channel) isdisplayed on the screen 122. The UI is overlaid on top of the content.

From step 1214, the control takes two paths. One to process Multiple,the other to Process Menu. In process Multiple, the processes discussedin FIG. 3, FIG. 4, FIG. 5, FIG. 6, FIG. 9, FIG. 10, and FIG. 11 areinvoked. As a result, Attribute Repository Structures 220 andPresentation Repository Structures 250, shown as Structure set 1220-1 inFIG. 12, are updated as the system learns about the user characteristicsand links them to content, products, advertisements, and services thatinterest him or her. The system stays in this loop changing display onscreen 122 as the processes indicate and updating structures 1220-1.

In process Menu, the UI is overlaid on content if the process Menu wasentered from step 1214 before the UI timed out. If the UI timed outpressing the Menu button on the remote control will overlay the UI onthe content being played. Next, the control goes to process Change User.The user points the remote control to the main-category region 710 inFIG. 7 and presses the up/down buttons on the remote causing themain-category items to scroll. All remote controls have the up/down orleft/right function implemented with buttons showing direction of themovement with arrows, track ball, scroll pad, or other such technology.When the Settings main-category is visible on the screen, the userpoints the cursor to that button and presses the OK button on the remotecontrol. The user has selected the Settings. The button will behighlighted. Using the PR Child Objects linked to this main-category,the system 100 will display the sub-category in region 730 in FIG. 7.The user will point the cursor to the sub-category region and scrollleft or right using the remote control until the User button is visible.Pointing the cursor on the User button in the sub-category and clickingon it will cause the level-1 sub-category to shift up to region 810 inFIG. 8 and a level-2 sub-category will appear in region 830 in FIG. 2.The level-2 sub-category would display buttons that shows the name ofthe users, one per button, who have a profile on the system 100. In step1216, the user sees if his or her name is in the level-2 sub-category byscrolling it left or right. If the user name is present, the user willpoint to it and click. The control will then go to the process SwitchUser. In this process, the user will be asked to enter a passcode (ifone exist), and after it is entered, the system will save currentobjects, for user Family, in the Structures 220 and 250 in Structure set1220-1 and then go to process User Selection. In this process, thesystem 100 switches to the new user, e.g., User 3, brings up the new ARand PR structures 220 and 250, shown as Structure set 1220-3 in FIG. 12,and then goes to the process Startup. If in step 1216, the user does notsee his or her name and wishes to create a new user, he points thecursor to New User button and clicks on it. Then in process Create User,a series of menus appear where the user is asked to enter the necessaryinformation, such as name, passcode, etc. As an example, he is assigneda name User 2. The settings for User 2 are saved. Next, the control goesto process User Setup where the default parameters, consisting ofdefault AR and PR structures 220, and 250, respectively, are set up forUser 2 in set 1220-2 in FIG. 12. Application 212 is linked to thesestructures. The control then goes to the Process Startup.

FIG. 13 show a UI Screen of FIG. 7 overlaid on a TV show 1310. Thebackground shows the content currently playing highlighted by themain-category button 1360 (TV), and the sub-category button 1380 (TVChannel1). The foreground shows the UI items. Collectively, the image1300 is the first screen of the UI in one exemplary embodiment.

Items in the UI can be selected by any of the means that currently existor are being developed. These include remote control devices withinterfaces such as infrared (IR), Bluetooth, or other wireless link,smart phones, tablets or other mobile devices running remote controlapplications, keyboards, mouse, voice control, and motion or gesturerecognition.

Action on a UI item is by moving the cursor to the item and thenclicking or pressing the OK button to execute the content, such asplayback, or pressing the Menu button to bring up the navigation andother command choices.

The UI components as seen in FIG. 13 and how the relate to the processesare discussed. Other layouts of the UI display are practical andexpected.

1340: Logo: Displays the service provider, manufacturer or customer'slogo.

1330: Time and Date.

1320: UserName: The name of the current user whose profile is being usedat that time. Profile implies the Attribute Repository and PresentationRepository structures as shown in FIG. 2. The UserName may consist ofFamily for a group of people, or individuals such as Mom, Dad, Joe, Sue,etc. Each one of these users will have their own AR and PR structuresand libraries containing content.

As discussed above, each profile can be accessed through a password, ifone exists, or other means such as face or fingerprint recognition thatmay become available in the future.

1360: Main Category Carousel—MCC (Music, Movies, InternetTV, TV, Radio,etc.): The carousel shows buttons that are represented by PR parentobject. The main-category can be static and fixed by the serviceprovider to limit the features of its offering. It can also be dynamicwhere new items can be added by the user. In the latter case, a newPresentation Repository parent object will be added to the list. Thecarousel effect is provided by a separate application and is well knownin the art. An example is the Radio button 1350 appearing smaller thanthe current highlighted button TV, 1360. The carousel can be rotated upor down to reveal more entries.

1370: Sub-Category Carousel—SCC (TV channels, etc.): The Sub-CategoryCarousel is to the right of the highlighted main carousel item (in thisexample the item labeled TV is the item highlighted in the MCC). Thenumber of elements in the SCC depends on the item highlighted in theMCC. Some MCC items have fewer SCC entries and others have a largenumber of SCC entries.

1380: TV Channel 1—A selection button in the sub-categories for themain-category 1360 (TV). The button is bigger and brighter than other TVChannel buttons; meaning highlighted, and indicates that this channel isproviding the content 1310 that is on the screen.

Advertisements

Advertisement is the primary source of revenue that drives thebusinesses of television and Internet. Content providers such as TV showproducers, movie studios, music producers, video distributors, andsearch engine providers, web portals, etc. derive all or a significantpart of the revenue from advertisements.

The system 100 with Attribute Repository Structures 220 and PresentationRepository Structures 250, FIG. 2, has been described above as havingthe ability to create knowledge of the users and that knowledge islinked to the information about content, products, advertisements, andservices that are of interest to the user. This ability enables thesystem 100 to display advertisements, promotions, merchants with deals,etc. of interest to the user at a time when he or she is most likely toremember, follow up, or close a transaction. The precise and targetedplacement of information before the user might be highly valued byadvertisers and might command a premium.

The primary form of advertisement in TV and through the Internet isthrough advertising campaigns. Advertising campaigns are when a contentprovider or an Internet service provider forms a business deal with aproduct or a service provider to promote their offerings throughadvertisements during a specific content and a specified period of time.The specific content could be a show or a movie that is rated high andso is expected to have a larger audience. Advertisements on such showscommand a higher price. Specified time would be a show run during primetime (e.g., 8PM to 11PM) when the potential number of viewers is usuallythe greatest, versus the show run very early in the morning when it isexpected to have much fewer viewers.

Vendors have been developing methods to make the advertisements moreeffective by trying to figure out how to “target” these advertisementstowards the audience so that they can command a premium from the contentcarriers (e.g., TV broadcasters).

There are issues with current advertisement and targeting methods thatthe system 100 addresses. The advertisements broadcasted through the TVprograms are intrusive and not targeted at all, and the one sent throughthe Internet are based on knowledge about the user one vendor hasassembled from a distance through tracking, history or other means.These advertisements are also intrusive and are coarsely targeted, if atall targeted. In addition, current systems do not have a good way toaccurately measure the effectiveness of advertisements over time.Effectiveness means that the viewer took action on the advertisement byfollowing it to see the product, such as going to a showroom to see anadvertised car, or close the transaction, such as buying the car.Companies such as Nielsen, AdWorks, and others have developed techniquesand technology (e.g. click count) in ad analytics which enables them tomeasure some ad effectiveness. But click count is not following theadvertisement or closing the transaction as described above. Adeffectiveness is the metric used by the product manufacturer or serviceprovider to pay the ad carriers. Ad carriers include content producersand content deliverers.

The system 100 and the algorithms described do not change the currentadvertising methods, should a vendor choose to deploy them, but providesa new model of “pulling the specific advertisement and relatedinformation” and presenting it to the user at the “right” time for amuch greater effectiveness. This new model might affect the way adcarriers, such as telephone, cable and satellite service providers, dobusiness with the content, product or service promoters, such asstudios, show producers, product manufacturers, etc. and allow the adcarriers to command a premium from advertisers whose advertisements aremore effective and proposing greater effectiveness strategies to thosewhose campaign-based advertisements are not producing the desiredresults.

Referring to FIG. 4 and process S4, details of which are provided inFIG. 6, in one embodiment the system 100 has identified AR object chainsthat link user characteristics with information about content, products,advertisements or services. After determining if the content beingplayed is live in step 450, the control proceeds to process D2. Thisprocess detects beginning of a campaign based ad that is sent by theprovider and the user is “forced” to watch. In process D3, the system100 locates the chains identified in process S4 and based on the weightscomputed in process S5, accesses the product or service search resultcompiled in process S5. The highest ranked search result and the highestweighted chain is the one which the system has identified as the one theuser has most interest in at this time. Using this chain of AR objects,the system locates the lowest level Child Object that has productinformation and/or link to the manufacturer website. The system 100 thenlocates the advertisement in the search results in process S5 and afteraccessing, begins playing it.

If the content is not live, the control then goes to process D5 wherethe system 100 locates the advertisement similar to the way describedabove for live content. The key difference between the two is that livecontent is played full screen or in the window by the system insertingthe ad in place of the live stream. For Internet streaming content or wepages that are not being played full screen, the areas around the maincontent has banner, stationery, or cursor sensitive advertisements. Thesystem can intercept and populate these areas of the screen withadvertisements it has located through the AR objects in the AR structure220 as the ones the user has interest in at this time.

The Attribute repository Structures 220 in FIG. 2 also shows AttributeFilter 234. As previously stated, Attribute Filter 234 are used tocommunicate specific information, defined by the object in the filter,such as AF-4, through the flow link 208 and network port 102 in FIG. 1to the service provider or the vendor. Thus, for each advertisementshown to the user, the system 100 can accurately communicate whatproduct or service was promoted. Since the system also manages other ARobjects in the chain based on user activity, AR objects associated withany activity that the user conducted for the promoted product would bein the AR object chain. Thus, if a product or a service promoted to theuser resulted in the user making a purchase on line, the system 100would know details of the merchant and the sale. Similarly, if the uservisited sites or did “more research” by visiting manufacturer or productweb sites, the system 100 would know that the user actively followed upon the promotion. This kind of information can also be sent to theservice provider or the vendor and would be a very accurate metric onthe effectiveness of the promotion.

Operation With Other Devices

The system 100 in FIG. 1 resides in a STB or a gateway that has largestorage 106 and sufficient computing power. A STB or a gateway typicallyis supported through a high speed Internet connection and a TV serviceprovided by a service provider (cable, telephone, or a satelliteservice) through a broadband network 102. The system 100 can also run ona mobile phone owned by the user where generally, the wireless broadband102 either does not have speeds comparable to cable or fiber broadbandand/or the cost of data service is very high.

The user can benefit from system 100 in the STB or the gateway as wellas when the system 100 is in a mobile device. The benefit in the mobiledevice is that the system 100 output to the screen 122 can provide theuser with information that matches his or her characteristics. Thisinformation is associated with content, products, advertisements, andservices and, possibly, location information, that the user can use.

An exemplary mobile device 1400 and 1500 is shown in FIG. 14 and FIG. 15having essentially all the same components of the system shown in FIG. 1and FIG. 2, respectively. However, mobile devices typically have a lesspowerful CPU 1404, smaller memory 1410, and smaller storage 1406compared to system 100 in FIG. 1. Other components shown in FIG. 14 aretailored to the networks, such as a radio for communicating with anaccess point or other service provider (not shown), and other devicesmobile device 1400 supports. The monitor 1422 is typically a touchscreen. In addition, the mobile device 1400 is much more sensitive topower consumption.

In FIG. 15, the one possible difference between system 100 (hereinafteralso referred to as an STB to differentiate it from a mobile device) andthe mobile device 1400 is in the Attribute Repository Structures 1520and in the Presentation Repository Structures 1550. The PR Structures1550 in the mobile device would be different if, (1) in one embodiment,the Application 1512 uses the UI that comes with the mobile device, (2)the UI has different main-,categories, such as a Location button, whichthe STB does not, and (3) the UI in the SIB and the mobile device is thesame except that the selection choices in sub-categories is reduced toconserve storage space. In all cases, the Presentation Filter 1564 inmobile device 1400 would define how the display should match the mobiledevice touch screen 1422.

In mobile device 1400, the AR Structures 1520 would have much fewerobjects in the database compared to a STB because of the small size ofstorage 1406. It is also possible that many parent AR objects 1522 andthe linked Child Objects (CO1-1, CO2-1) could exist on both the STB andthe mobile device. The existence of an AR object in the STB and themobile device indicates that the user characteristic, information oncontent, product, advertisements or services, identified by thisspecific AR object, is useful to both the STB and the mobile device. Itis also possible that the STB could have many more AR objects in its ARStructure 220, whereas mobile device 1400 could have other AR objects inits AR Structures 1520 that the STB would not have. An example ofdifferent AR objects in mobile device 1400 would be places (locations)and things associated with those places.

The algorithms in FIG. 3, FIG. 4, FIG. 5, and FIG. 6 discussed how thesystem 100 forms the AR object chains and how they are weighted toidentify the user characteristic and the information on product andservices that most interest the user. In the mobile device 1400, theApplication 1512 runs independently and forms its own AR object chainsto better match the user characteristics with the information it haslearned. Thus, if the AR structures 1520 have AR objects associated withthe location, following process S4 in FIG. 5, the Application 1512 willform its own weighted chain that will match the user characteristicswith location and the product and services the user has interest in. Theonly thing remaining to do is to synchronize AR objects in the mobiledevice with the AR objects in the STB so that the STB can learn newthings about the location and what the user had interest in. If the STBis the hub device of the user domain, it may want to use the AR objectsthe mobile device has created but would not have a need to update it.Extending this concept further, if there are other user owned devicessuch as PC, tablets, and other mobile phones, they all need to besynchronized with the AR objects they share with the hub (STB) device.Note that the simplest example of shared AR object would be the ParentAR object 222 in the STB and 1522 in the mobile device, as they describethe user characteristics and the user is the same on all the devices.

In order to perform synchronization, the first step is for all devicesto recognize each other and be aware that they belong to the same ownerregardless of the means through which they communicate with each other.The means could be a USB connection, a TCP/UDP network connection,public wired and wireless networks, and so on. Mechanisms exist toidentify devices using universal plug and play (uPnP), DLNA, and othersincluding many proprietary methods.

It was previously stated that all external communication from a hubdevice with system 100 is managed through the Attribute Filter 234 inFIG. 2. Communication to service provider, advertisers, and merchants iscontrolled by the Attribute Filter 234 as described earlier. In the samemanner, communication with other devices on private and public networkand owned by the user is also controlled through the Attribute Filter234. The Attribute Filter 234 contains device objects, AF-1, AF-2, etc.Some of these objects are reserved to identify all the devices that areowned by the user and in communication on the network, also referred toherein as the user's domain, such as the mobile device 1400. There istypically one device object per device. Contained within the AttributeFilter for a device, such as AF-4, are states that describe rules ofcommunication with other devices in the user's domain. To understand themethod better, the addressing and identification of devices uses adesignation of the type: HomeDomain.devicetype.ID; where HomeDomainmight be the unique name of the owner account which could be provided bythe service provider; devicetype might be the name and model of thedevice such as iPhone V, iPad 2, Lenovo T61P PC, Sony PS3, etc. andprovides the device's characteristics (CPU, memory, storage); and IDmight be a unique ID to identify an individual device.

When a new mobile or satellite device needs to be added to the user'sdomain, with STB or the gateway as the hub device, the Application 212(FIG. 2) and the default structures, or a subset thereof, are installedon the newly added device as new application software to register thenew device with the hub 100. In the mobile device 1400, this would beApplication 1512 and the structures 1520 and 1550. Included in thestructures is the Attribute Filter 1534 that includes the reference tothe Attribute Filter object, as an example, AF-4, providing addressingand protocol information for the hub device 100. The newly added deviceis then capable of issuing and responding to commands from the hubdevice 100 (here the STB or the gateway) provided the hub device statestored in Attribute Filter, e.g. AF-4, in the mobile device allows it.

Flow 1600 in FIG. 16 illustrates how as device is brought in the user'sdomain and how the state of the device in the Attribute. Fitter of thehub device 100 and the mobile device 1400 changes to allow therespective device to communicate with each other. There are threecolumns shown with label Hub Device 1602, Commands/Responses 1604, andthe Mobile Device 1606. In the embodiment, Mobile Device is used as anexample. Any other device can be brought into the user domain in thesame way. When the mobile device is not in the user domain, its state inthe Attribute Filter 234, in AF-3 for example, in the hub device isState Free, as shown in process H1. A State Free prevents the hub devicefrom sending or receiving any functional commands, such as send orreceive data exchange AR objects, etc., to the mobile device. Only statequery or state transition commands can be exchanged. Unless a devicestate is “State Acquired” for both devices as described below,functional communication cannot take place between the devices.

Consider as an example that a mobile device 1400 powers up and is readyto communicate with the hub device 100. In this instance, process M1shows the state of the hub device in the mobile device Attribute Filter1534, here object AF-4, as State Free (default). By default, all otherdevices Attribute Filter objects are in State Free when a device, suchas the mobile device, first power up and is read to enter communicationwith those devices. Process M1 in FIG. 16 shows this state and the stepsthat are followed by the mobile device to acquire the hub device. First,it checks that the hub device state is State Free. In order to check thehub device state, the mobile device 1400 first checks its AttributeFilter 1534, object AF-4, to validate that it is permitted tocommunicate with the hub device 100. Validation includes checkingHomeDomain, nub device ID and the device state. If the entry is valid,the mobile device 1400 uses the appropriate network protocols to connectitself to the user subscribed wireless network that also provides itwith access to the Internet. Using the wireless network, the Application1512 in the mobile device 1400 issues an “Acquire(HomeDomain.devicetype.ID)” command to the hub device 100.

The “Acquire (HomeDomain.devicetype.ID)” command from the mobile device1400 is received by the hub device 100 in process H1. After completingthe receive command, the hub device 100 goes to process H2. The hubdevice first checks if the sending device is permitted to communicatewith it by validating its AF object, AF-3 as an example, in theAttribute Filter 234. Validation requires checking the HomeDomain nameand the devicetype, such as a mobile device 1400 of type iPhone 5. Ifthe check succeeds, the hub device then checks if the incoming commandis consistent with the stored state of the mobile device, which is StateFree (from process H1). As an example, if the mobile device AF-3 entryin the hub device 100 indicates that the mobile device is in a “StateAcquired” state and another Acquire command is received, then the huhdevice might consider the mobile device as not operating correctly andgenerates an exception. Exception may cause the hub-device to initiate aprogrammatic recovery of the mobile device. If the state of the mobiledevice and the incoming command is consistent, the hub device 100authenticates the mobile device's particulars in AF-3 and transitionsthe device state from State Free to State Acquired. A response is thenissued to the mobile device of Acquire Success. Upon receiving anAcquire Success response in process M2, the mobile device 1400transitions its hub device Attribute Filter object AF-4 in 1534 to StateAcquired.

Similar to the Acquire command, a “Free (HomeDomain.devicetype.ID)”command is also provided that when issued by the mobile device causes itto transition the AF-4, as in the above example, for the hub device in1534, to a State Free. Upon receiving a “Free(HomeDomain.devicetype.ID)” command, the hub device 100 transitions themobile device Attribute Filter, AF-3 in the above example, in AttributeFilters 234, to a State Free. A response of Free Success is issued bythe hub device 100 which the mobile device may or may not receive, e.g.when powered off or the network is experiencing long delays. If themobile device is powered off, then upon powering up the hub state in itsAttribute Filter in 1534 would default to State Free, as described abovein process M1, FIG. 16. Receiving a Free Success response will notchange anything in the mobile device as it is in the correct state. Ifthe mobile device is not powered off, and it does not receive the FreeSuccess response, it will continue to show hub device in the StateAcquired in AF-4, in Attribute Filter 1534. It may issue commands to thehub device which will receive a response of state error. At this time,the mobile device can be powered off and powered on, or the hub devicewill initiate state check and recovery for the mobile device.

In order to avoid issues with state conflicts, i.e., where the statesstored in the hub and the mobile device do not agree, the Acquire andFree commands are executed by devices in their entirety before the nextAcquire or Free command is processed.

The State Acquired state of the Attribute Filter for a target deviceenables the device to carry out synchronization when needed. If thestate is State Free, no operational communication, such as datatransfers and non-state changing commands, can be initiated by theApplications 212 and 1512 and, hence, no synchronization or othercommunication supported by the Application can take place.

As previously stated, Attribute Filters 234 also provide information onthe protocol that a device supports. If the mobile device established aconnection with the hub device using the IP address or the USB, the hubdevice would know how to communicate with the mobile device (using an IPaddress based protocol or the USB).

Similar to the “UI category tag” state defined earlier in the AR objectsin AR structures 220 of FIG. 2, another state is defined in the ARobject called State Original and State Copy. The new states establishthe “ownership” of a particular AR object in an environment where the ARobject can exist in more than one device. AR object exists in more thanone device when the user characteristic or the information on content,product, advertisement or services (properties) represented by the ARobject is used by both devices. In such an environment, as would be thecase if the user has more than one device in the user domain, the needarises to manage such AR objects between a hub device 100 (such as STBor a gateway) and a mobile device 1400. The AR object contains two typesof information. The first type of information is the propertyinformation, such as keywords, URLs, etc. that defines what this ARobject represents. The second type of information is meaningful only inthe context of the device. Such information includes visit count,forward and backward links, etc. The properties information whensynchronized is beneficial in that it reduces analysis and AR objectcreation computation in the mobile device and makes the mobile deviceprovide the most up to date interest-centric information about the userto the huh 100. Rules are defined that uses the two states, StateOriginal and State Copy, in the AR objects to manage synchronization sothat both devices can have the latest AR object properties. The device'scontext-sensitive information, such as visit count or links, isgenerally not shared between the devices as such information ismeaningful only to the device in which the AR object is stored.

An AR object in AR structure 220 in the hub device or AR structure 1520in the mobile device with a State Original indicates that it is most upto date and, if in need of updating, would be the one to be updated. AnAR object with a State Copy indicates that this is a copy of the ARobject with to State Original that resides in another device. An ARobject with State Copy can be read as many times as needed but cannot beupdated. By definition, all objects in the hub device when created haveState Original and all objects installed in the mobile or satellitedevices when first installed have State Copy.

The rules for managing the State Original and State Copy in the ARobjects can be illustrated in one embodiment as a flow 1700 in FIG. 17.As in FIG. 16, there are three columns shown with label Hub Device 1702,Commands/Responses 1704, and the Mobile Device 1706. Process SH1 in flow1700 shows that the hub device 100 has the Attribute Repository objectexemplary AR-2 in AR 222, in FIG. 2, with State Original. The hub deviceis the owner of object AR-2. The mobile device 1400 also has an objectAR-2 stored in AR 1522 (FIG. 15) with the state of State Copy. Themobile device can use the object AR-2 stored in AR 1522 for comparison,analysis, and to form chains. The mobile device 1400 cannot modify theobject AR-2, in 1522, with properties information, such as URL. Devicecontext sensitive information can be modified as described above.

In flow 1700 in FIG. 17 and in process SM1, the mobile device has thecopy of AR-2 object as shown. During content analysis, the mobile devicedetermines that the properties information in Attribute Repositoryobject AR-2 in 1522 needs to be updated, e.g. adding a new URL. To doso, the mobile device 1400 broadcasts a Request Original AR-2 command1710 to all devices that are in the State Acquired in the AttributeFilter 1534 in FIG. 15. In this embodiment, the broadcast 1710 comprisesmultiple single commands sent to each device that is in the StateAcquired in the Attribute Filter 1534, in the mobile device 1400. FIG.17, process SM1, shows three commands 1710 going to three differentdevices, x, y, and hub, that are in the State Acquired in the AttributeFilter 1534. More or fewer commands 1710 might be sent depending on thenumber of other devices the mobile device 1400 is aware of having theState Acquired. Further, the broadcast 1710 can be a single commanddepending upon implementation of the communication protocol betweendevices.

Of the commands 1710 sent by the mobile device, the hub device 100receives one of these commands. In process SH1, the AR-2 object in thehuh device in 222, is in the State Original. The hub device transitionsthe AR object AR2 from the State Original to State Copy. If this AR iscached in any other place in the hub device, such as cache memory (notshown), it is purged. The hub device 100 then explicitly responds to themobile device 1400 with a reply 1720 of Response Original AR-2.

In process SM2, the mobile device 1400 receives the Response OriginalAR-2, shown as 1720 in FIG. 17. The mobile device then transitions theobject AR-2 state stored in AR 1522 from a State Copy to a StateOriginal. The mobile device 1400 would be updating the propertiesinformation in the object AR-2 in AR 1522. After the AR-2 object statehas changed to State Original, the properties information in AR-2 isthen updated. All the context sensitive information of the device 1400stored in object AR-2 in AR 1522, such as visit count, links, etc. arepreserved. The mobile device 1400 goes to state SM3 where it issues abroadcast command 1730 of Send Copy AR2 to all devices that are in theState Acquired in the Attribute Filter 1534 in FIG. 15. In thisembodiment, the broadcast 1730 is one command at a time to the devicesthat are in the State Acquired but can be a single command dependingupon implementation of the communication protocol between devices.

In process SH2, the hub device 100 receives the Send Copy AR-2 command1730. Since this command is a send of updated properties information inthe AR-2 object, the hub device updates the stored object AR-2 in AR 222with the newly received properties information. The device contextsensitive information is retained as described above.

The FIG. 17 illustrates rules that can be stated as follows:

(1) When creating a new AR object, its state is State Original. Thecreating device must broadcast the State Copy of this AR object so thatother devices can update the properties information of the AR object. Tokeep track of the AR objects, AR objects in common but on differentdevices have a common reference. As embodied here, an AR object has areference that can be constructed in many ways and is not limited to aphysical memory address but might be a logical address as desired.

(2) When a device does not have an AR object and wants to check if anyother device has the same AR object, it can broadcast a request for acopy of the AR object. When received, the state of the stored AR objectin the requesting device will be State Copy.

(3) If a device wants to update an AR object that has a state of StateCopy, it must request the AR object with State Original that mightreside in any of the devices in the user's domain. After update, thedevice owning the AR object broadcasts a copy of the updated AR objectso if any other device has the same AR object, it can be updated withnew information. The updated AR object in such devices will have a stateof State Copy.

(4) If a device has an AR object with State Copy and it wants to use itwithout an update of properties information, the AR object will continueto have a state of State Copy.

(5) Lastly, if two devices created new AR objects with states of StateOriginals, then when they receive copies of the AR object per rule one,conflict is resolved in favor of the newest AR object or, when theobjects are created at exactly the same time, the object created on thehub device wins.

If the mobile device is operating such that the items in the AttributeRepositories have not changed (meaning they are Copies), then nosynchronization will be needed. This means that no new items arelearned. Existing AR objects are used to provide user interest centriccontent and information.

If the mobile device creates a new AR object, its state is StateOriginal. The new AR object can be sent to the hub device with a SendCopy command when the mobile device is in the vicinity of the hub deviceand on a high-speed network and synchronization is taking place. The hubdevice will receive the AR object with a state of State Copy.

During the course of normal operation on a mobile device, a user mayencounter an AR object that has links pointing to content. If thecontent is not present on the mobile device, a “Request Content Copy”command (not shown) will be issued to all devices. Devices that have acopy of the content will respond with a suitable result. The mobiledevice can then issue a “Transfer Content” command to a device that willresult in the target device establishing a data transfer connection withthe requesting device and transferring the content thereto.

Similar to the Request Content Copy command described above thattransfers content from one user device to another, a command is providedthat will “pass through” content from a subscription a user may havethrough the hub device but is restricted from other devices for whateverreason. Following a successful connection with the hub device and asign-in by the mobile device into one of the subscriptions that the usermay have on the hub device (STB), the subscribed content can betransferred from the provider, through the hub device, and transferredto the mobile device. For example, if the user is in a foreign countryand issues a command “Request Subscription ID Connect” to the huh, thehub device uses the ID to tune to a TV channel or logs-in to a web-basedcontent provider, if so requested. After a successful connection in thehub device (STD) to a TV channel or a log-in to the web-based contentprovider (subscription), the hub will issue a “Transfer Content” commandto the mobile device. Once acknowledgement is received, the contentreceived by the hub device may then be “passed through” to the mobiledevice. If the hub was receiving a streamed content from the source (TVor web), it will be “streamed” to the mobile device over, for example, asecure device-to-device link.

As described above, the exemplary embodiment is for TV viewing at homewhere the user benefits from the system 100. The system 100 cansimilarly be used to support commercial businesses by changing the usageclassification to target a business such as a dentist, doctor, lawyer,auto dealership, warehouse, etc. where the user needs to knowinformation pertaining to their products, services, subscriptions,customer feedback, etc. A UI can be defined that supports a particularbusiness. The Seed 224 in the Attribute Repository Structures 220 can beprovided that contains the terms aligned with this business. All otherprocesses and structures described previously will stay the same. Sincethe invention is adaptive in nature, if deployed, it will learn from thebusiness and will begin to make targeted recommendations.

While the exemplary embodiments of the present invention have beendescribed with respect to processing blocks in a software program,including possible implementation as a digital signal processor,micro-controller, or general-purpose computer, the present invention isnot so limited. As would be apparent to one skilled in the art, variousfunctions of software might also be implemented as processes ofcircuits. Such circuits might be employed in, for example, a singleintegrated circuit, a multi-chip module, a single card, or multi-cardcircuits pack.

As used herein in reference to an element and a standard, the term“compatible” means that the element communicates with other elements ina manner wholly or partially specified by the standard, and would berecognized by other elements as sufficiently capable of communicatingwith the other elements in the manner specified by the standard. Thecompatible element does not need to operate internally in a mannerspecified by the standard.

Although the subject matter described herein may have been described inthe context of illustrative implementations to process one or morecomputing application features/operations for a computing applicationhaving user-interactive components the subject matter is not limited tothese particular embodiments. Rather, the techniques described hereincan be applied to any suitable type of user-interactive componentexecution management methods, systems, platforms, or apparatus.

The present invention can be embodied in the form of methods andapparatuses for practicing those methods. The present invention can alsobe embodied in the form of program code embodied in tangible media, suchas magnetic recording media, optical recording media, solid statememory, floppy diskettes, CD-ROMs, bard drives, or any othernon-transitory machine-readable storage medium, wherein, when theprogram code is loaded into and executed by a machine, such as acomputer, the machine becomes an apparatus for practicing the invention.The present invention can also be embodied in the form of program code,for example, whether stored in a non-transitory machine-readable storagemedium, loaded into and/or executed by a machine, or transmitted oversome transmission medium or carrier, such as over electrical wiring orcabling, through fiber optics, or via electromagnetic radiation,wherein, when the program code is loaded into and executed by a machine,such as a computer, the machine becomes an apparatus for practicing theinvention. When implemented on a general-purpose processor, the programcode segments combine with the processor to provide a unique device thatoperates analogously to specific logic circuits. The present inventioncan also be embodied in the form of a bitstream or other sequence ofsignal values electrically or optically transmitted through a medium,stored magnetic-field variations in a magnetic recording medium, etc.,generated using a method and/or an apparatus of the present invention.

It will be further understood that various changes in the details,materials, and arrangements of the parts which have been described andillustrated in order to explain the nature of this invention might bemade by those skilled in the art without departing from the scope of theinvention as expressed in the following claims.

The invention claimed is:
 1. A method comprising: a) selecting apresentation parent object in a presentation object database having aplurality of presentation parent objects therein; b) identifying apresentation template corresponding to the selected presentation parentobject, the identified presentation template having keywords therein andhaving one or more corresponding lower level presentation templates; c)searching an attribute object database to identify an attribute parentobject therein that matches one or more of the keywords from theidentified presentation template, each attribute parent object linked toone or more attribute child objects; d) searching the one or moreattribute child objects linked to the attribute parent object identifiedin step c) to identify one or more of the attribute child objects thatmatches one or more keywords in the one or more lower level presentationtemplates corresponding to the identified presentation template; e)sorting the one or more attribute child objects identified in step d)based on at least one trait associated with each of the attribute childobjects; f) creating one or more presentation child objects from the oneor more attribute child objects sorted in step e); g) linking togetherthe one or more presentation child objects created in step f) in thesame order as the one or more attribute child objects sorted in step e);h) linking together the one or more presentation child objects linkedtogether in step g) with the selected presentation parent object; and i)displaying on a display at least the selected presentation parentobject.
 2. A method comprising: a) selecting a presentation parentobject in a presentation object database having a plurality ofpresentation parent objects therein; b) searching a plurality ofpresentation templates to identify a presentation template that matchesone or more keywords in the selected presentation parent object, theidentified presentation template having keywords therein and having oneor more corresponding lower level presentation templates; c) searchingan attribute object database to identify an attribute parent objecttherein that matches one or more of the keywords from the identifiedpresentation parent template, each attribute parent object linked to oneor more attribute child objects; d) searching the one or more attributechild objects linked to the attribute parent object identified in stepc) to identify one or more of the attribute child objects that matchesone or more keywords in the one or more lower level presentationtemplates corresponding to the identified presentation template; e)sorting the one or more attribute child objects identified in step d)based on at least one trait associated with each of the attribute childobjects; f) creating one or more presentation child objects from the oneor more attribute child objects sorted in step e); g) linking togetherthe one or more presentation child objects created in step f) in thesame order as the one or more attribute child objects sorted in step e);h) linking together the one or more presentation child objects linkedtogether in step g) with the selected presentation parent object; and i)displaying on a display the selected presentation parent object andselected ones of the presentation child objects linked to the selectedpresentation parent object; wherein the presentation parent and childobjects are displayed as individual buttons on the display, and thepresentation parent and child objects specifying functionality of thecorresponding button when the button is selected by a user.
 3. Anon-transitory machine-readable medium having encoded thereon programcode, wherein, when the program code is executed by a machine, themachine implements a process comprising the steps of: a) selecting apresentation parent object in a presentation object database having aplurality of presentation parent objects therein; b) identifying apresentation template corresponding to the selected presentation parentobject, the identified presentation template having keywords therein andhaving one or more corresponding lower level presentation templates; c)searching an attribute object database to identify an attribute parentobject therein that matches one or more of the keywords from theidentified presentation template, each attribute parent object linked toone or more attribute child objects; d) searching the one or moreattribute child objects linked to the attribute parent object identifiedin step c) to identify one or more of the attribute child objects thatmatches one or more keywords in the one or more lower level presentationtemplates corresponding to the identified presentation template; e)sorting the one or more attribute child objects identified in step d)based on at least one trait associated with each of the attribute childobjects; f) creating one or more presentation child objects from the oneor more attribute child objects sorted in step e); g) linking togetherthe one or more presentation child objects created in step f) in thesame order as the one or more attribute child objects sorted in step e);h) linking together the one or more presentation child objects linkedtogether in step g) with the selected presentation parent object; and i)displaying on a display coupled to the machine at least the selectedpresentation parent object.
 4. The method of claim 1 wherein steps a)-i)are repeated for each presentation parent object located in thepresentation object database.
 5. The method of claim 1 wherein in stepf) the one or more presentation child objects are created in accordancewith a corresponding one of the one or more lower level presentationtemplates.
 6. The method of claim 1 wherein selected ones of thepresentation child objects linked to the selected presentation parentobject are also displayed in step i).
 7. The method of claim 1 whereinthe keywords include phrases.
 8. The method of claim 1 wherein thesearching of the attribute child objects in step d) uses keywords fromall of the lower level presentation templates corresponding to theidentified presentation template.
 9. The method of claim 1 wherein theone or more presentation child objects created in step f) are stored inthe presentation object database.
 10. The method of claim 1 wherein theone or more attribute child objects are stored in the attribute objectdatabase.
 11. The method of claim 1 wherein the searching of attributechild, objects in step d) uses keywords from a specified number of theone or more lower level presentation templates corresponding to theidentified presentation template, the specified number being stored inthe identified presentation template.
 12. The method of claim 1 whereinthe presentation template is identified in step b) by the step of:searching a plurality of presentation templates to identify thepresentation template that matches one or more keywords in the selectedpresentation parent object.
 13. The method of claim 1 wherein the atleast one trait is related to a number of links each attribute childobject has to other attribute parent or child objects.
 14. The method ofclaim 1 further comprising the step of: displaying content on thedisplay; wherein the selected presentation parent object overlays thedisplayed content on the display.
 15. The method of claim 1 wherein instep i) selected ones of the presentation child objects linked to theselected presentation parent object are displayed on the display, thepresentation parent and child objects are displayed as individualbuttons on the display, and the presentation parent and child objectsspecifying functionality of the corresponding button when the button isselected by a user.
 16. The method of claim 1 wherein in step f) onepresentation child object is created for each attribute child objectsorted in step e).
 17. The method of claim 2 wherein steps a)-i) arerepeated for each presentation parent object located in the presentationobject database.
 18. The method of claim 2 wherein in step f) the one ormore presentation child objects are created in accordance with acorresponding one of the one or more lower level presentation templates.19. The method of claim 2 wherein all of the presentation child objectslinked to the selected presentation parent object are also displayed instep i).
 20. The method of claim 2 wherein the keywords include phrases.21. The method of claim 2 wherein the searching of attribute childobjects in step d) uses keywords from all of the lower levelpresentation templates corresponding to the identified presentationtemplate.
 22. The method of claim 2 wherein the one or more presentationchild objects created in step f) are stored in the presentation objectdatabase.
 23. The method of claim 2 wherein the one or more attributechild objects are stored in the attribute object database.
 24. Themethod of claim 2 wherein the searching of attribute child objects instep d) uses keywords from a specified number of the one or more lowerlevel presentation templates corresponding to the identifiedpresentation template, the specified number being stored in theidentified presentation template.
 25. The method of claim 2 wherein theat least one trait is related to a number of links each attribute childobject has to other attribute parent or child objects.
 26. The method ofclaim 2 further comprising the step of: displaying content on thedisplay; wherein the selected presentation parent object overlays thedisplayed content on the display.
 27. The method of claim 2 wherein eachof the presentation templates include display property information, andthe step of displaying the presentation parent and child objects is madein accordance with the display property information in the identifiedpresentation template and corresponding lower level presentationtemplates, respectively.
 28. The method of claim 2 wherein when anbutton is selected by the user, at least one of a plurality of functionsassociated with the selected button is performed depending on how thebutton is selected by the user.
 29. The method of claim 2 wherein instep f) one presentation child object is created for each attributechild object sorted in step e).
 30. The method of claim 2 wherein instep g) the one or more of the presentation child objects are linkedsequentially by level and when a button corresponding to a presentationchild object is selected by a user, a button corresponding to apresentation child object having a level lower than the presentationchild object corresponding to the selected button is displayed on thedisplay.
 31. The non-transitory machine-readable medium of claim 3wherein steps a)-i) are repeated for each presentation parent objectlocated in the presentation object database.
 32. The non-transitorymachine-readable medium of claim 3 wherein in step f) the one or morepresentation child objects are created in accordance with acorresponding one of the one or more lower level presentation templates.33. The non-transitory machine-readable medium of claim 3 whereinselected ones of the presentation child objects linked to the selectedpresentation parent object are also displayed in step i).
 34. Thenon-transitory machine-readable medium of claim 3 wherein selected onesof the presentation child objects linked to the selected presentationparent object are also displayed in step i).
 35. The method of claim 3wherein in step i) selected ones of the presentation child objectslinked to the selected presentation parent object are displayed on adisplay, the presentation parent and child objects are displayed asindividual buttons on the display, and the presentation parent and childobjects specifying functionality of the corresponding button when thebutton is selected by a user.
 36. The method of claim 6 wherein all ofthe presentation child objects linked to the selected presentationparent object are also displayed in step i).
 37. The method of claim 6wherein each of the presentation templates includes display propertyinformation, and the step of displaying the presentation parent andchild objects is made in accordance with the display propertyinformation in the identified presentation template and correspondinglower level presentation templates, respectively.
 38. The method ofclaim 15 wherein when a button is selected by the user, at least one ofa plurality of functions associated with the selected button isperformed depending on how the button is selected by the user.
 39. Themethod of claim 15 wherein in step g) the one or more of thepresentation child objects are linked sequentially by level, and when abutton corresponding to a presentation child object is selected by auser, a button corresponding to a presentation child object having alevel lower than the presentation child object corresponding to theselected button is displayed on the display.
 40. The method of claim 28wherein the functions associated with the selected button are 1)displaying other presentation child objects linked to the presentationparent or child object corresponding to the selected button, 2) causingcontent specified by the presentation parent or child objectcorresponding to the button to appear on the display, and 3) performingan action in accordance with the corresponding presentation template.41. The method of claim 35 wherein in step g) the one or more of thepresentation child objects are linked sequentially by level and when abutton corresponding to a presentation child object is selected by auser, a button corresponding to a presentation child object having alevel lower than the presentation child object corresponding to theselected button is displayed on the display.
 42. The method of claim 38wherein the functions associated with the selected button are 1)displaying other presentation child objects linked to the presentationparent or child object corresponding to the selected button, 2) causingcontent specified by the presentation parent or child objectcorresponding to the button to appear on the display, and 3) performingan action in accordance with the corresponding presentation template.